Interrupt management

So, next on the list is interrupt management. We have already discussed  it briefly before, from a technical point of view, when I was trying to figure out how to devise an abstract interrupt handling workflow that works on a wide range of computer architectures. Here, the goal is to explore interrupt handling in more details, and noticeably see some caveats which I have not explored before about the notion of primary interrupt source and its incompatibility with a lean kernel design.

Primary interrupt sources

Ideally, interrupt-aware computer hardware would work this way : the CPU has one “interrupt pin”, and one of its I/O ports is connected to an “interrupt bus”. When the interrupt pin receives a signal, the CPU automatically fetches an “interrupt ID” from the interrupt bus, which uniquely represents the independent piece of hardware that has generated the interrupt (may be as simple as the device’s GUID in an UEFI world). Control is then sent to the kernel, which is able to check if a driver has registered a handler for that interrupt ID, and run that handler asynchronously if it is so. Drivers are called in a minimal time, only the kernel can fail before the driver gets its interrupt, everyone is happy.

Sadly, real life is not ideal, especially when we’re talking about computer architectures. In practice, hardware is connected to the CPU using a very deep tree of buses, each following its own unique design philosophy which takes hundreds of pages to describe and millions of lines of code to support. As an example, on x86, when a mere USB pen drive sends an interrupt to say that some data transfer operation is done, all the CPU gets is an interrupt signal from the PCI(e) bus controller. It then takes a PCI bus driver to tell that the signal originates in fact from a USB host controller, so that the USB driver can in turn kick in and dive through the unholy mess of controllers that makes the USB standard in order to determine that yes, it’s that tiny pen drive over there that is responsible for the interrupt. The driver that is directly associated to the USB pen drive may then be awakened, and ordered to do its thing (presumably notify the file system services in a device-independent way).

The lesson to learn from this is that real-world kernels cannot know where exactly interrupts hitting a CPU pin come from and call the proper driver directly, unless they set out to support every standard and nonstandard computer bus on every architecture they support. Running that amount of code in kernel mode would be, in my opinion, an unnecessary source of bloat, inflexibility, and security flaws. So at the other extreme, one would be tempted to suggest a mode of operation where the kernel only aims at considering interrupts that are directly coming at CPU pins, and leave the additional layers of interrupt redirection to more qualified user-space bus drivers. Yes, but…

The scheduler problem

We happen to have this very important kernel component called a scheduler. To work properly on modern machines, it requires two things : a programmable clock interrupt, and a way to communicate with other CPU cores (wake them up, give them work to do). On most architectures, this requires the kernel itself to manage a few layers of interrupt redirection, such as the APICs in the x86 family.

This is a bad thing because it means that a big chunk of arch-specific functionality will have to be put into the kernel, and that an arch-specific interface for accessing all of this functionality will need to be defined if it turns out to be needed. What is probably the best option to start with, though, is to adopt a “black box” philosophy, where the system only broadcasts the inputs to the interrupt redirection layers that it manages, and sets those layers it manages in the most straightforward way so that a combination of CPU interrupt signals and I/O may be used to unambiguously identify where an interrupt comes from. Things like an x86-specific APIC configuration interface may easily be added to the Interrupt Manager in the future if there happens to be a need for it one day.

In this design, it is critical that the interrupt manager’s interface provide everything that is required for user-mode bus drivers to understand what the kernel’s interrupt sources are, since they aren’t a well-defined standard object like a CPU interrupt vector in themselves. For every architecture, a standard “interrupt descriptor format” must be devised, allowing drivers to acquire knowledge of the available interrupt sources, just like Intel’s MP specification allows a monolithic x86 kernel to know how ISA IRQs are routed into IOAPICs’ input pins. For more formal interrupt manipulation, a numeric “interrupt ID” should do wonders. Which means that there should be a mechanism to know if a given interrupt source is present (and if so what its interrupt ID is).

“Plug and play” issues

To conclude this article, since we’re talking about a kernel area that deals directly with drivers, I believe that it’s important to say something about the way the kernel decides that a specific driver should be run or not. How does the kernel know, without implementing gigantic specs, what kind of hardware is present in the computer, in order to choose which hardware to run in a “plug and play” fashion ?

The answer is simple : it doesn’t, because it doesn’t need to. The goal of this kernel, once again, is to provide the basic building blocks of the process abstraction, so that every other part of the OS can run in an isolated process from the point of view of the CPU it executes on. From this point, everything is implemented in user space. The kernel can be ordered through boot options to run the driver for a hardware whose presence can be assumed on all target hardware (such as the ISA keyboard and VESA framebuffer on current x86 desktops). It can as well be ordered to run a special “plug and play” driver which probes interrupt sources, MP configuration tables, and other stuff to find out which top-level hardware and bus drivers must be loaded.

All the kernel has to do for this to work is to provide a secure interface for (privileged) user mode code to tap into its capabilities and other privileged areas of the computer (such as memory-mapped peripherals and CPU Model Specific Registers). This interface must be simple, future-proof, and as architecture-agnostic as possible in order to reduce the amount of code duplication across supported architectures. Generally speaking, coming up with a beautifully clean interface to kernel-mode services and managing said services is the job of the kernel, whereas using them is the job of user-mode processes.

As a conclusion…

This article was quite difficult to write, because it had to deal with arch-specific notions at an abstract, arch-independent level, without diluting the core message too much in the way. I hope that it remained clear enough though. Here is a short summary of the key points :

  • Modern computer architectures are way too complex for detection of primary interrupt sources in the kernel to be a good idea.
  • Their conception, however, prevents the kernel from neglecting all the layers of interrupt redirection that exist beyond the CPU’s interrupt vectors.
  • As such, this kernel will adopt a compromise in which the minimal amount of interrupt redirection that is required for kernel functionality to work will be implemented, but not more.
  • Said functionality will be implemented, whenever architecturally possible, in the most straightforward way allowing input interrupts to be separated from each other, so that the unnecessary interrupt redirection layers become invisible to software in a “black box” approach.
  • When a black box redirection of interrupts is not possible, a standard kernel interface to configure the kernel-managed redirection layers will be added to the Interrupt Manager. This outcome, however, must be avoided at all cost.
  • It is not the job of the kernel to load and set up the appropriate drivers for the detected interrupt sources. This job will be left to static boot-time kernel driver modules, or more flexible but slower run-time hardware probing.
  • The kernel, however, must provide any interface that is necessary for drivers to find out where the interface that they look for is, or for “meta-drivers” to examine the list of interrupt sources (and other arch-specific data) in order to determine which drivers should be loaded.

And that’s it, thanks for reading !

2 thoughts on “Interrupt management

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s