Abstracting the interrupt interface

[TylerH]

I'd like to ask the community for ideas on how to abstract away the concept of an interrupt handler into a more general concept of an exception handler for a specific exception.

A central goal of my kernel is to have the same interface on all platforms, for both drivers and regular processes. (Actually, this just follows from the real goal of having only regular processes. I want to get rid of the concept of a driver, replacing it with the concept of a privileged process; privileged in the sense that a privileged process can use interfaces others can't, not that a privileged process runs in a privileged processor mode.) I want to have an interface by which a privileged process can register a handler for a given exception, like a disk interrupt, for example.

What I don't have the perspective to know is how much work this will require in the kernel. What I don't want to end up with is a bunch of semi-drivers in the kernel for configuring interrupts to work with my exception interface. How much hardware dependent stuff must be done to determine what interrupt handles what (or to configure the hardware to use a given interrupt)? Is it really feasible to try to provide an interface that allows processes to register disk/network/other hardware handlers without having a lot of hardware dependent code in the kernel?

Thanks for your time,
Tyler

[Rudster816]

Exceptions and interrupts are entirely different things, and should be handled completely different from each other.

What you have to decide is what the driver should be able to control over how the interrupt is handled. E.g. rather or not it exclusively "owns" the IRQ it wants, the interrupts priority (and hence its vector), rather or not the driver provided ISR (interrupt service routine) acknowledges the interrupt, or if this is automatically done by the kernel, etc.

The first thing you need to do is maintain a list of which Bus IRQs correspond to which interrupt vectors, because the device driver should only have to 'request' an IRQ without having to know which interrupt vector it corresponds to. Obviously you will have initialize the PIC\IOAPIC(s)\etc first in order to be able to handle interrupts in the first place, so you should know these mappings, or be able to set them.

Then you'll have to install a basic IDT entry that eventually will call the device drivers ISR. An easy way to do this is to make an assembly stub that can easily be modified to call a given address which should correspond to a C function. My x64 template looks something like this in NASM

Code: Select all

%macro PUSHAQ 0
	push r15
	push r14
	push r13
	push r12
	push r11
	push r10
	push r9
	push r8
	push rbp
	push rsi
	push rdi
	push rdx
	push rcx
	push rbx
	push rax
%endmacro
	
%macro POPAQ 0
	pop rax
	pop rbx
	pop rcx
	pop rdx
	pop rdi
	pop rsi
	pop rbp
	pop r8
	pop r9
	pop r10
	pop r11
	pop r12
	pop r13
	pop r14
	pop r15
%endmacro

isr_template:
	PUSHAQ	
	mov rdi, rsp
	xor rsi, rsi
	nop
	mov rax, 0xAAAAAAAAAAAAAAAA
	mov si, 0xCC
	call rax
	POPAQ
	iretq
        times 25 db 0

You can then make a copy of the machine code and easily edit the calling address and interrupt vector through a simply C structure like this one:

Code: Select all

typedef struct __attribute__((__packed__)) isr
{
	char Pad1[32];
	unsigned long IsrFunction;
	char Pad2[2];
	unsigned char IsrVector;
	char Pad3[53];
} isr_t;

You just have to look at the binary output that NASM generates and modify the struct as needed. An easy way to do this is to just assemble your template and have NASM output it as a flat binary. Then just open it up in a hex editor and look for whatever dummy address\vector you put in.

You should probably call your own C function that that will in turn call the drivers ISR with better information (since both the vector and registers are irrelevant for device interrupts). This will also allow you to 'daisy chain' interrupts, allowing multiple drivers to have their own ISR for the same interrupt.

One other thing I can think of right now is that you're running on a NUMA system, you should have the capability to force the ISR to only run on the processor's memory the driver is loaded into in order to ensure that the ISR is executed as quickly as possible.

[Combuster]

The necessary interface for an interrupt is just giving a driver the message that it's device has caused an IRQ, and the driver must call back on that without sleeping when or if it had made the device to stop pulling it's line.

The necessary interface for an exception is either telling the driver it's stupid, or providing the whole architectural state. The latter demands awareness from the driver in question. The former is sufficient for anything that isn't a debugger.

[gravaera]

Yo:

I'd like to ask the community for ideas on how to abstract away the concept of an interrupt handler into a more general concept of an exception handler for a specific exception.

...Is it really feasible to try to provide an interface that allows processes to register disk/network/other hardware handlers without having a lot of hardware dependent code in the kernel?

Thanks for your time,
Tyler

1. You shouldn't allow ISRs to be installed on "exception" vectors, and your IRQ-controller management code should never allow an interrupt controller pin to be mapped to an exception vector. For the i8259 this would mean reprogramming the controllers' vector bases, as you've seen in basically any article that mentions ISA IRQs, and for the IO-APICs that would mean making sure that you never program an exception vector into the table.

2. Not too sure what the question here is: it's feasible in that it's necessary, and by its very nature a kernel will have hardware dependent code somewhere. You just need to be sure to separate it well and glue it together neatly. I would say an offhanded "yes" to this question . Though you could have ISRs in separate address spaces with their drivers if you want something like that, but I'm sure you know that it'll slow things down :O

--Peace out
gravaera

[TylerH]

Okay, my wording was ambiguous (or just wrong). By exception, I was referring to any synchronous execution transfer. Like a synchronous call to the driver or an IRQ.

[TylerH]

1. You shouldn't allow ISRs to be installed on "exception" vectors, and your IRQ-controller management code should never allow an interrupt controller pin to be mapped to an exception vector. For the i8259 this would mean reprogramming the controllers' vector bases, as you've seen in basically any article that mentions ISA IRQs, and for the IO-APICs that would mean making sure that you never program an exception vector into the table.

Yeah, I wasn't. The idea is actually not to allow them to install to vectors at all, but to register a handler with the kernel for a specific reason, with the kernel taking care of which vector that reason corresponds to.

2. Not too sure what the question here is: it's feasible in that it's necessary, and by its very nature a kernel will have hardware dependent code somewhere. You just need to be sure to separate it well and glue it together neatly. I would say an offhanded "yes" to this question . Though you could have ISRs in separate address spaces with their drivers if you want something like that, but I'm sure you know that it'll slow things down :O

Okay, this is where I think my ignorance is causing me confusing in the planning stages. I was thinking that I would have to have code specific to a particular piece of hardware (like a HDD or something). I'd prefer to limit the code in my kernel to dealing only with the CPU, memory, and related controllers.

I'm actually planning on allowing the interrupt to jump directly into the address space of the handler, but I've still got to work out the details on how I would do this. I think it would require me to use hardware task switching (or to trust the handler to save regs). I'm not going to trust the handler to save regs.

[gerryg400]

I'm actually planning on allowing the interrupt to jump directly into the address space of the handler, but I've still got to work out the details on how I would do this. I think it would require me to use hardware task switching (or to trust the handler to save regs). I'm not going to trust the handler to save regs.

No need for HW task switching. My API looks like this

Code: Select all

    int anvil_intr_create(int __bus, int __inum, void *(*__isr)(void *), void *__arg);
    int anvil_intr_destroy(int __id);
    int anvil_intr_disable(int __id);
    int anvil_intr_enable(int __id);

A driver process (which is no different from any other process except it has IO privity) calls anvil_intr_create to install the handler. The hanlder would be a C function taking and returning "void *". Multiple handlers can be installed. When an interrupt occurs the kernel does all the normal interrupt stuff like saving state, locking the vector to prevent re-entry, and then calls (literally with a 'call' instruction) each installed handler in turn. It obviously also switches to the memory context of the process .

The isr handlers run in ring 0 even though they are in the memory context of the process and live in the lower half with the process code.

[qw]

Rudster816 - Shouldn't you pop the registers in reverse order? Now the contents of r15 end up in rax.

[Rudster816]

I must have overwritten that source file with an older version, because I've fixed that two or three times already.....

[qw]

We need an opcode for "execute the next 16 instructions in reverse order"

[gerryg400]

You should use mov instead of push or pop. Then the order doesn't matter as much.

[TylerH]

Do all platforms/processors use numbered interrupts? Not being sure of that was the original reason I considered it a good idea to base my interrupt handler registration off of purpose instead of interrupt number. If I were to implement my interrupt handler registration interface by allowing processes to register handlers by specifying the exact interrupt number, I would want the API to be the same for all platforms, thus all platforms would need to use numbered interrupts.

[Owen]

Do all platforms/processors use numbered interrupts? Not being sure of that was the original reason I considered it a good idea to base my interrupt handler registration off of purpose instead of interrupt number. If I were to implement my interrupt handler registration interface by allowing processes to register handlers by specifying the exact interrupt number, I would want the API to be the same for all platforms, thus all platforms would need to use numbered interrupts.

No. On ARM, for example, all the CPU knows about are "IRQ" and "FIQ" (plus various processor-local exceptions, such as SWI and various aborts).

Rather than an "IRQ number", pass some form of handle around. In my case, that handle looks a lot like a Windows event object - and is indistinguishable from other application-created event objects, which enables things like userspace IRQ multiplexers to be created. When a bus driver (e.g. the PCI driver) enumerates a device, it hands its driver a collection of event objects - if the driver is sharing IRQs, each will be one "pin" on a virtual IRQ multiplexer device (which can be implemented in kernel or in userspace); if the driver is not sharing, each will correspond directly to a real IRQ.

Event objects in the signalled state correspond to masked IRQs, and those in the unsignalled state correspond to unmasked IRQs. Whenever an IRQ is raised, signal the event object; whenever an application resets the event object, unmask the IRQ.

You might want some drivers in kernel space for performance. For example, I would consider implementing things like the whole PC APIC system (including ACPI-level "logical interrupt pins") as a kernel driver, along with at least part of the PCI bus driver, and drivers for things like ARM's Nested Vectored Interrupt Controllers.

[TylerH]

Do all platforms/processors use numbered interrupts? Not being sure of that was the original reason I considered it a good idea to base my interrupt handler registration off of purpose instead of interrupt number. If I were to implement my interrupt handler registration interface by allowing processes to register handlers by specifying the exact interrupt number, I would want the API to be the same for all platforms, thus all platforms would need to use numbered interrupts.

No. On ARM, for example, all the CPU knows about are "IRQ" and "FIQ" (plus various processor-local exceptions, such as SWI and various aborts).

Rather than an "IRQ number", pass some form of handle around. In my case, that handle looks a lot like a Windows event object - and is indistinguishable from other application-created event objects, which enables things like userspace IRQ multiplexers to be created. When a bus driver (e.g. the PCI driver) enumerates a device, it hands its driver a collection of event objects - if the driver is sharing IRQs, each will be one "pin" on a virtual IRQ multiplexer device (which can be implemented in kernel or in userspace); if the driver is not sharing, each will correspond directly to a real IRQ.

Event objects in the signalled state correspond to masked IRQs, and those in the unsignalled state correspond to unmasked IRQs. Whenever an IRQ is raised, signal the event object; whenever an application resets the event object, unmask the IRQ.

You might want some drivers in kernel space for performance. For example, I would consider implementing things like the whole PC APIC system (including ACPI-level "logical interrupt pins") as a kernel driver, along with at least part of the PCI bus driver, and drivers for things like ARM's Nested Vectored Interrupt Controllers.

That (event objects) is exactly what I was thinking. I'll look more into how Windows uses event objects and maybe see if there is an analogous object in Minix, since I'm aiming for a microkernel design. Thanks for the info.

[rdos]

Having a uniform way of doing IRQ handlers is not hard to implement. The difficult thing is to get the physical IRQ number in the PIC/IO-APIC for a given device, and this has no single solution. For a PCI-device, you can get the PCI int PIN from the PCI interface, but you cannot get the IRQ number (unless legacy PIC is used) without ACPI. For ISA, there usually is no method other than ACPI or auto-detect. Additionally, getting IRQs from ACPI requires AML. Another factor is that PCI IRQs can be shared, meaning that there need to be logic to detect if an IRQ is for the handler or not, and to chain.