Multi or single instance services?

[OSwhatever]

There is an interesting design choice how you can make you services work the operating system. In a system where everything are user processes, services, drivers, everything you choose how to design the interfaces. Either you can create a new process for every service. Let's say you have a driver and several similar devices, then you can create driver that can host all the devices in one process, or you can create a new process for each of those devices.

Such a design choice really impacts how you write the interfaces throughout the system. The advantages and disadvantages are obvious, one process per instance consume more memory. Several instances per process makes the system more robust as if one instance fails, it will not affect the others.

How would you go in such a design choice? Can you create the interface so that this is completely transparent to the end user process?

[dchapiesky]

BSD RUMP kernels....

RUMP has the added advantage of allowing said services to actually be running on remote hardware....

http://rumpkernel.org/

or just go for a channel based approach like google's fuscia/magenta OS

where all services communicate through "channels" (glorified ring buffers) to the kernel

https://en.wikipedia.org/wiki/Google_Fuchsia

https://github.com/fuchsia-mirror/magenta

overboard examples with increased security include such things as L4 Microkernels and "Partitioned" OS's which define strict APIs between services

Also remember that on x86_64 at least, the TSS can specify an arbitrary ring # - you don't have to stick to ring 0,1,2,3 ... you can have as many as you like as long as your security architecture tracks dependencies correctly.... With appropriate ring buffers and memory allocation (i.e. magenta) you can use the hardware to help enforce the API rules

[dozniak]

, one process per instance consume more memory. Several instances per process makes the system more robust as if one instance fails, it will not affect the others.

It's also easier to DOS a system if it doesn't have proper resource accounting/allocation by spawning more requests than server is able to serve and causing OS to spawn more server processes to handle the load, like a fork-bomb.

I'd go for single-threaded server frontend which takes requests using a select-like mechanism and then either dispatches it to thread inside the same process or spawns off a new process if certain conditions are met. This is more flexible, lets the server control the load and presents only one uniform interface to clients.

[xenos]

Another option would be to have a single, but multi-threaded server process, with one thread for each device it operates. Then this process would only have a single memory image for its code, thus reducing memory usage, but could serve requests for different devices simultaneously on a multi-processing system. Also if the number of threads is constant (one per device), a request to the server would not spawn a new thread or process, and thus not trigger a "fork bomb" on DOS (but of course a DOS would still keep the driver busy).

[Korona]

My OS takes a pragmatic approach. It uses a microkernel design and separates most but not all drivers into different user-mode processes. When separating functionality would hurt performance too much however I tend to prefer better performance over greater separation. For example each block device driver (e.g. ATA, USB bulk-only, virtio) runs in a separate process but a single driver instance handles all devices of the same type. The FS code runs inside the block device driver's process because I want to avoid context switch chains like application -> fs driver -> block device driver -> fs driver -> application.

I only use a single thread in each driver processes and I'm using asynchronous I/O everywhere. I could extend the design to one thread per CPU but then I would have to ensure that the code was thread-safe. Using more than one thread per CPU (e.g. one thread per device) does not really make sense if you're using asynchronous I/O.

[OSwhatever]

My OS takes a pragmatic approach. It uses a microkernel design and separates most but not all drivers into different user-mode processes. When separating functionality would hurt performance too much however I tend to prefer better performance over greater separation. For example each block device driver (e.g. ATA, USB bulk-only, virtio) runs in a separate process but a single driver instance handles all devices of the same type. The FS code runs inside the block device driver's process because I want to avoid context switch chains like application -> fs driver -> block device driver -> fs driver -> application.

I only use a single thread in each driver processes and I'm using asynchronous I/O everywhere. I could extend the design to one thread per CPU but then I would have to ensure that the code was thread-safe. Using more than one thread per CPU (e.g. one thread per device) does not really make sense if you're using asynchronous I/O.

That's an interesting approach. The file system code, is it loaded as a shared library into the block device driver?

[Korona]

That's an interesting approach. The file system code, is it loaded as a shared library into the block device driver?

Yes. The file system code resides in a shared library.

Drivers and applications communicate via a capability based IPC mechanism. I use lightweight message queues to inform processes that some operation is complete. The fast path in that code (i.e. when there are messages available) does not even require a syscall.

[Boris]

My OS will allow both.
You create domains which can be used only as memory spaces, but you can feed them with many IPC services. You can put unrelated apps together if they dont collide in memory ( both in symbols and heap . Stack is fine, my stacks have all their own memory space, per thread)