Microkernel and file descriptors

[ichigo]

Hello Everyone!

For those of you that are developing a microkernel-based operating system, how're you handling the case of the stdin, stdout and stderr file descriptors (in libc, vfs, or kernel)? Does your VFS server open them by default for all the processes? or does a process message the VFS server to initialize them (which could be done during libc initialization)?

I'm more interested to know how do you handle this situation during early boot. In my case, I've a process manager which will message the VFS when a process forks. However, during boot (and before the process manager is loaded) the kernel loads the VFS, and few other more essential servers (like FS, and disk drivers) before the process manager. I'm still thinking about the best way to inform the VFS server about the processes loaded by the kernel (not the process manager). I think it's important to have those file descriptors open as soon as possible so that early servers and drivers can log important messages.

I would love to hear about your solution to this problem.

Thank you!

[nullplan]

I'm not a microkernel guy, but isn't the usual solution to this to just inherit file handles? So the kernel launches whatever servers it wants with no open file descriptors, and then when it is ready, the first process gets to open the default devices for stdin, stdout, and stderr, and then those are always bestowed onto child processes, until those change the FDs.

The only possible challenge here is to maintain PID 1 for init, but you can create the init process without FDs and make it wait for the other processes to become available before opening the devices.

In other words, I would just not treat the first three FDs as in any way special, is my point.

[qookie]

I'm not a microkernel guy, but isn't the usual solution to this to just inherit file handles?

This is probably the best solution if you're trying to be POSIX-compatible, considering not only stdin, stdout, and stderr, but all file descriptors are inherited on fork, and they survive across exec unless they specifically were marked as CLOEXEC.

As a sidenote: opening stdin, stdout, and stderr in PID 1 is what's done in managarm as well, and init can open a console in that state since it's started by the POSIX server, not the other way around (since outside of that server, there is no concept of a PID anyway).

[linguofreak]

I'll also note that a file descriptor provided by the kernel or VFS to libc, and a file descriptor as provided by libc to other userspace code in the same process, don't have to be the same thing. With a microkernel, it's possible to imagine an architecture where pipes, files on disk, and tty input are handled by different servers, in which case libc file descriptors would be entirely a runtime construct internal to the process: libc would have a file descriptor table which would then specify which server provides the stream or file associated with a given descriptor and what handle needs to be used in communicating with *that server* about the corresponding file. A given server would then not actually know if a given file handle that it was providing to a given process was being used as a standard stream for that process or not.

[AndrewAPrice]

I'm not making a POSIX compatible kernel, so my processes interact with the outside world via Services and RPCs. Services are interfaces, and any process can implement a service and register it in the kernel. So, a process that wants to read or write to disk will query the kernel for implementations of 'File Service'*, and it's possible it's not yet started (so I have a synchronous query-and-wait call), and once we have a reference to a File Service, we can call FileService->ReadFile...

For terminal input/output (where does STDOUT go? where does STDIN come from?) the program will look up Terminal Service which has read/write methods. Intercepting STDOUT/STDIN and redirecting it to the passed in Terminal Service is something I handle in my C library->system binding. My OS supports std::cout, printf, and normal C/C++ file handling this way.

Of course, my OS isn't complete, but this is the plan:
If a process just calls Terminal Service out of the blue, it'll open a new terminal window, and Terminal Service will remember PID 1234 talks to window 4. If PID 1234 launches a new process (let's call it PID 5678) PID 1234 can tell Terminal Service that PID 5678 can share window 4.

In a more complicated and flexible world, I'd support sandboxing. So, when a parent program launches a child program, it could tell the kernel to launch it with a given "sandbox". When the child looks up Terminal Service, the parent (which might be an IDE that's running terminal commands in a docked panel) could intercept the syscall query for Terminal Service and return its own implementation.

My design is going to anger people who want to stay true to the POSIX. Do what makes you happy!

[Ethin]

I'm not making a POSIX compatible kernel, so my processes interact with the outside world via Services and RPCs. Services are interfaces, and any process can implement a service and register it in the kernel. So, a process that wants to read or write to disk will query the kernel for implementations of 'File Service'*, and it's possible it's not yet started (so I have a synchronous query-and-wait call), and once we have a reference to a File Service, we can call FileService->ReadFile...

For terminal input/output (where does STDOUT go? where does STDIN come from?) the program will look up Terminal Service which has read/write methods. Intercepting STDOUT/STDIN and redirecting it to the passed in Terminal Service is something I handle in my C library->system binding. My OS supports std::cout, printf, and normal C/C++ file handling this way.

Of course, my OS isn't complete, but this is the plan:
If a process just calls Terminal Service out of the blue, it'll open a new terminal window, and Terminal Service will remember PID 1234 talks to window 4. If PID 1234 launches a new process (let's call it PID 5678) PID 1234 can tell Terminal Service that PID 5678 can share window 4.

In a more complicated and flexible world, I'd support sandboxing. So, when a parent program launches a child program, it could tell the kernel to launch it with a given "sandbox". When the child looks up Terminal Service, the parent (which might be an IDE that's running terminal commands in a docked panel) could intercept the syscall query for Terminal Service and return its own implementation.

My design is going to anger people who want to stay true to the POSIX. Do what makes you happy!

Why not have the graphics service remember window 4? Then you can create terminals without tying them to windows. I'm not endorsing the Unix philosophy... I just see an opportunity for flexibility here. If tying terminals to windows is an optional thing, you can have pseudoterminals and such, all managed through the terminal service.