Newcomer's microkernel doubts

[iansjack]

I'm just saying that if I wanted the easy way to manage an HDD I could just do some int 13h or something and call it a day...

And I'm saying that you couldn't.

I appreciate that your purpose is to learn. That's exactly why I am suggesting other ways that you might like to learn. There are more options than monolithic or micro - it's interesting to explore all of them. That way you get to understand the choices that OS designers make.

[iansjack]

@iansjack: Perhaps I am wrong, but I think you missed the OP's point..

Or perhaps you missed my point?

In my first reply in this thread I pointed to a way to run device drivers in user space. (I didn't explain it n ABC detail, because I'm more in favour of a pointing the way approach than spoon-feeding.) And then later I went on to point out that there is more than one way to skin a cat. It's worth exploring other possibilities such as hybrid monolithic/micro kernels. After all, that's the way a lot of very successful OSs do it. I'm trying to expand the OP's horizons rather than just strictly answering the original question.

The business about BIOS calls is a red herring. You and I know that that is not "the easy way out" in any sense, unless you just want to write a trivial, seriously crippled OS.

[frabert]

You and I know that that is not "the easy way out" in any sense, unless you just want to write a trivial, seriously crippled OS.

That's the point I've been trying to make, I see no sense in making a trivial, crippled OS, and I very much understand how using BIOS routines would make my project end up as such. But I guess we can both agree than calling a BIOS subroutine is easier that writing a robust ATA(PI)/AHCI driver, right? Also, the reason I want to make all my drivers in userland is not one of practicality, or success - otherwise, I might as well have stopped doing osdev at all. It's because I find it more fascinating. Don't get me wrong, I know there are lots of ways to do the same things, but to me putting more code into the kernel to let application do IO does not sound exciting, that's all. It is practical, though, and this much I understand very well.

BTW, thanks for the pointers, they did definitely help, and I appreciate the hints at the different ways to approach the problem

[OSwhatever]

but to me putting more code into the kernel to let application do IO does not sound exciting, that's all. It is practical, though, and this much I understand very well.

Yes, absolutely. Apart from the already known benefits of having drivers in user space (debugging, stability, faster development etc), from a OS developer view I actually think it is beneficial to "get out to user space" as quickly as possible. This encourage you to early on create some kind of IPC infrastructure and also working in user space will give you hints how your kernel interface should look like. By making real programs you will better understand what needs to be done. Drivers are excellent here as they don't use that many resources (other services) which means you can start with these early.

[iansjack]

I never suggested putting the drivers in the kernel. The only difference between your plans and what I suggested merited consideration is that the drivers run as separate processes in supervisor mode rather than user mode. Solves a lot of problems with (IMO) no significant disadvantages.

Repeat - they are not part of the kernel.

[Korona]

That sounds like the worst of all worlds to me. iansjack, could you explain where the benefits of such a design are? After all, you still need IPC as in a microkernel (which introduces complexity) and the isolation between drivers is only slightly stronger than in a monolithic kernel.

[iansjack]

OK. It's a terrible idea.

[frabert]

[You can have] the drivers run as separate processes in supervisor mode rather than user mode. Solves a lot of problems with no significant disadvantages.

I'd have to disagree here, since a bug can potentially take down the entire kernel, right? I'm not really sure how actually powerful process separation can be in supervisor mode, I never actually researched it...
BTW, one of the things I'd like to achieve is a kernel that cannot be taken down by faulty drivers. That's the idea, at least...

[OSwhatever]

I'd have to disagree here, since a bug can potentially take down the entire kernel, right? I'm not really sure how actually powerful process separation can be in supervisor mode, I never actually researched it...
BTW, one of the things I'd like to achieve is a kernel that cannot be taken down by faulty drivers. That's the idea, at least...

Yes, user mode drivers shouldn't run i supervisor mode for the very reasons you mentioned. Only, reason I can think of is that you run them in supervisor mode during development time before you have the resource management ready.

When you have total process isolation for drivers, why not use it fully instead for stability.

[Korona]

Note that not even a microkernel (without hardware assistance) can guarantee that faulty driver cannot take down the system. Most (all?) modern devices allow DMA to arbitrary physical addresses, which can be used to break the isolation. Thus, for true isolation you need a IOMMU. Real IOMMUs, however, do not support per-PCI-function isolation. Instead, multiple PCI devices are often placed into the same IOMMU group and thus share the same address space.

Of course, in practice, microkernels do offer better isolation than monolithic kernels and it's much harder to hack a system using device DMA than it is to do the same via plain memory accesses.

[OSwhatever]

however, do not support per-PCI-function isolation. Instead, multiple PCI devices are often placed into the same IOMMU group and thus share the same address space.

In these cases there also the possibility to load several drivers into the same process. Drivers can be shared objects (DLLs) which are loaded into a framework process. There are commercial examples of this in microkernels out there.

Loading several drivers into the same process can have several benefits. Faster interface between framework and driver, shared buffering solutions, shared DMA solutions. Of course if one driver fails it will sink the entire process but this is often a acceptable compromise.

[Korona]

Yes. Even if a IOMMU is not available, putting drivers into microkernels is a very useful idea. In my kernel, the FS drivers are actually implemented in a library that every block device driver links against. Similarly, there are libraries to provide the boilerplate code for graphics drivers and for virtio drivers. In the future, I plan to implement drivers for network interfaces as "plugins" that are loaded by a networking daemon. This way packets never have to cross process boundaries even if they need to be routed from one network interface to another.

[Schol-R-LEA]

Yes. Even if a IOMMU is not available, putting drivers into microkernels is a very useful idea.

As a historical aside, the primary reason for the development of the micro kernel concept in the late 1970s was to demonstrate that a stable system was feasible with no hardware memory protection or separation of user and supervisor modes at all, by applying the Actor model's message passing approach to support conventional procedural (i.e., not specifically OOP) software , as well as counter the assertion that semaphores were absolutely necessary for effective mutual exclusion.

The key term here is 'stable'. Pretty much every previous multitasking OS that didn't require some kind of additional hardware support for separating user and supervisor modes had serious stability problems (as did many that did have it). The main exception was THE, and that only managed it because Dijkstra designed it as a batch system with strict limits on what was allowed to run on it.

[dseller]

Yes. Even if a IOMMU is not available, putting drivers into microkernels is a very useful idea.

As a historical aside, the primary reason for the development of the micro kernel concept in the late 1970s was to demonstrate that a stable system was feasible with no hardware memory protection or separation of user and supervisor modes at all, by applying the Actor model's message passing approach to support conventional procedural (i.e., not specifically OOP) software , as well as counter the assertion that semaphores were absolutely necessary for effective mutual exclusion.

The key term here is 'stable'. Pretty much every previous multitasking OS that didn't require some kind of additional hardware support for separating user and supervisor modes had serious stability problems (as did many that did have it). The main exception was THE, and that only managed it because Dijkstra designed it as a batch system with strict limits on what was allowed to run on it.

Interesting, I am currently working on a new kernel that uses the actor model. Did it actually implement the whole model, or just the message passing aspect of it?

[Schol-R-LEA]

Interesting, I am currently working on a new kernel that uses the actor model. Did it actually implement the whole model, or just the message passing aspect of it?

Oh, I am sorry if I was unclear; THE wasn't a micro-kernel, being a ring-layered system instead, and was from about ten years before the micro-kernel idea really caught on with researchers. Indeed, it was designed before the entire concept of a 'kernel' per se existed.

THE was Dijkstra's demonstrator system to show that semaphores were a workable method of mutual exclusion on a system-wide scale, and while the mainframe it ran on (the Electrologica x8) didn't have a separate supervisor mode or hardware memory protection, it did have a Test-and-Set instruction which provided the atomic locking required for consistent semaphores. It succeeded in that, but the limitations imposed on the system to make it feasible without hardware memory protection or a supervisor mode - no interactive user mode, a small fixed number of total processes, programs only allowed to use a single process, memory sizes fixed at compile time, etc - somewhat undermined the point, and contrary to Dijkstra's intent of demonstrating a minimum-practical method, helped convince other OS designers that additional hardware support beyond the atomic instructions he was advocating was absolutely necessary for a non-toy system.

According to Wicked-Pedo, the first system to follow a kernel design was the RC 4000 Mutiprogramming System, which was around the same time as THE, and the current article argues that it could be considered a 'microkernel' despite the fact that it doesn't really fit the later definitions of that term - indeed, the term 'kernel' itself (or the equivalent 'nucleus', which was the more common one in the 1970s) was only introduced years later. The idea of a micro-kernel which uses message passing for IPC and separates drivers from the kernel image didn't really come until the mid to late 1970s, I think.

While they borrowed the idea of message passing from the Actor model, that was pretty much all most microkernels took from it in that period as far as I know.

Also, note that since the point was to reduce the need for hardware protection (important with the increasing use of 12-bit and 18-bit minis such as the PDP-8, and even more so with the appearance of 8- and 16-bit microprocessors such as the 8080, 6800, 6502, 8086, and 68000), the idea that the driver processes were 'running in user space' didn't really apply, since there was no 'supervisor space' to be separate from - the separation came from putting the drivers in their own processes, and having all communication between them and the user applications be moderated by the kernel through message-based IPC. The idea was that as long as the kernel could interrupt a hanging process and kill it if it were unresponsive, the system could remain up even if an application or driver was corrupted. This still didn't solve certain problems, such as the lack of privileged instructions, so at least some microkernel systems at the time placed limits on what code could be emitted for application or driver processes, and used either a different toolchain or a set of toolchain switches when compiling and linking the kernel.

Note that there were a number of multitasking OSes for some of the early home and small business computers ('microcomputers') in the late 1970s and early 1980s, including CHAOS (Clairmont High School Advanced OS, which was a timesharing system running on an Altair with a lot of extra memory and I/O boards, it was a unique installation with no successors but apparently it got a lot of press around 1977; no connection to the Linux fork of that name), OS-9 (a 6809 OS made famous by the Tandy CoCo and later ported to the 68K), and MP/M (a multitasking version of CP/M first released in 1979, and at least one source I've read which was meant to be the main PC OS, with CP/M-86 being a stopgap while it was ported), and while the Wikipedia article doesn't mention it, I seem to recall reading that later versions of UCSD Pascal p-system (which I've heard from other sources was the one IBM wanted as the PC's main OS, so there's some disagreement on this point) were multitasking as well. MS-DOS was already a retrograde design when it came out, and the main reason it caught on was because the alternatives cost three to ten times as much.