Reinventing Unix is not my problem

[rdos]

Having devices appear in the filesystem doesn't mean that they are necessarily limited to the traditional Unix APIs that copy. Memory mapping is also an option (even under conventional Unix), and under UX/RT there will also be read/write-type APIs that operate on kernel message registers as well as ones that operate on a shared buffer. As long as the filesystem API is sufficient there should be no need to add extra primitives alongside it, since anything can be relatively easily implemented on top of unstructured I/O streams and memory regions. For services where read/write-type APIs are inconvenient for client code to use directly, higher-level library interfaces can be provided on top of the file-based API. UX/RT will provide higher-level library wrappers for every service except those for which direct use of read/write is the only good realization (which is unlike Linux and other conventional Unices where a lot of special files lack any high-level wrapper even though they could use one).

Regardless of method, it's basically impossible to implement a zero-copy file API with both the shared buffer interface (which is obvious) and with memory mapping. This is because file clusters might be smaller than the page size, which means you cannot memory map the file without allocating an intermediate buffer. What is needed is that the kernel must be able to memory map the file in chunks of it's underlying sector size, and the application must be able to handle that the mapping might not be continous rather in chunks. With traditional memory mapping, it's typically the application that defines where the mapping should reside, which won't work with zero-copy. Unless you create an overly complex disc buffer architecture.

Both spawn()/CreateProcess() and traditional fork() suck as process creation primitives IMO because they both do way too much. UX/RT will instead have an "efork()" ("eviscerated/empty fork") primitive that creates a completely empty process in a non-runnable state. This will return a VFS RPC file descriptor to the child process context, and the parent will set up the environment of the child process by calling the same APIs that it uses to control its own environment (all APIs that manipulate process state will have versions that take a process context, and it will also be possible to switch the default context for the traditional versions that don't take one). To start the process, the parent will call either the traditional exec() to start the process run another program, or a new exec()-type call that takes an entry point (this of course will assume that appropriate memory mappings have been set up), after which the VFS RPC connection will become a pidfd and cease to accept any new calls from the parent to change the child's state (it will be possible to use it to wait for an exit status from the child). This eliminates the overhead of fork() for spawning other programs and the overhead of spawn()/CreateProcess() for spawning copies of the same program. It also preserves the ease of manipulating the child's context that comes with fork() and doesn't require a primitive with a whole bunch of different arguments and flags to try to include every possible context modification like spawn()/CreateProcess(). Both fork() and spawn() will be easy to implement as library functions on top of efork().

My CreateProcess basically creates an empty context similar to CreateThread. For instance, the kernel debugger is a kernel process in an isolated context with no user context. To actually run a user application is another API that starts with CreateProcess and then does exec() in kernel space. Yet, my CreateProcess cannot be used to implement fork. Fork requires another way to initiate the process context and so uses another API.

I think user accounts should still be present since you may have multiple people using the same system, but they shouldn't be the primary security model. UX/RT will completely eliminate the traditional root/non-root security model of legacy Unix and replace it with per-process lists of permissions to specific files or entire directories (which may specify to take the permissions from the filesystem instead of providing explicit permissions) and a role-based access control system built on top of them. The setuid and setgid bits on executables will no longer have any effect and will be replaced by a database of permissions.

I'm not building a desktop OS for multiple users, rather I'm more focused on embedded systems and servers/data collectors.

It will be easy enough to support such an interface under UX/RT, with a server exporting one or more character- or message-special files for sending commands and configuration, and a memory shadow file covering the buffer to which the analyzer is streaming data. Alternatively if there's no need to multiplex access to the device and you want to avoid any overhead of having a separate server, the application could just access it directly by mapping the files for the device's physical memory regions (along with setting up I/O port permissions if applicable) and setting up an interrupt handler, since UX/RT will be a pure microkernel system and will allow full user access to I/O devices (subject to permissions of course).

I suppose you (as the OS creator) could easily add this, but could somebody else do it without changing or adding interfaces to your OS? This is the problem in Windows/Linux. In my design, somebody could actually add a device driver for this that interacts with PCI to find & initiate the device and that exports a set of APIs for user level. This would not require any kernel modifications. The only thing it would require are some private gate numbers that could just be picked at the end.

[rdos]

Supporting user account when logging into to servers (or when acting as a server yourself) is ok. It's the user account model on the local computer that is outdated. Once I have my ext4 VFS and NTFS VFS I can easily break the security model of both Windows and Linux by booting my own OS on the machine and then being able to access EVERYTHING since I won't support the access control embedded into the filesystem. This is just security by complexity that isn't worth anything.

The classic user account model on local computers has never been designed to protect against somebody who has physical access to the machine (e.g. you can reboot it and boot another OS): it protects users from unintentionally accessing (or writing) data they shouldn't and it allows per-user customization. For example: me and my wife share some home machines. We don't need "security" against each other on the local accounts, but having different accounts is still necessary: each one has his own account with different settings, files and wallpaper. Also, we cannot unintentionally delete each other's files. All of this it's not broken: it's a valid use case, but I agree that nothing to do with security.

Sure, that's valid for a desktop OS. However, Unix forces this model on ANY machine, regardless if it is a desktop OS or an embedded system.

If you want to protect data against people who have physical access to the machine, you have to use encryption. But actually, even that is not enough because experts use any sort of techniques to work around encryption like reading data directly from the memory chips after turning off the machine in a "rough" way, trying to recover the keys. So, if you really want to protect against people who have physical access to the machine, all the time, that's a security nightmare. You probably have to buy special machines designed against such attacks and, still, have to limit yourself a lot on the software you're using. I'm no security expert to tell more, but you get the idea..

Adding a custom FPGA on the PCIe bus could do wonders. It basically can read everything from memory while the OS is executing without the OS having an idea what is going on.

[vvaltchev]

Sure, that's valid for a desktop OS. However, Unix forces this model on ANY machine, regardless if it is a desktop OS or an embedded system.

Yes, that's true because UNIX was never meant for embedded systems. At the time UNIX was invented microcontrollers had no OS at all. Btw, my project, Tilck, is an example of a UNIX-like kernel without support for accounts. You have just root, but there's still some permission checking, breaking apart from UNIX: for example, you can use chmod() to make a file read-only and, if you try to open it for writing, you'll get an error, even if you're root. That's useful to help preventing unintentional damage.

Adding a custom FPGA on the PCIe bus could do wonders. It basically can read everything from memory while the OS is executing without the OS having an idea what is going on.

Yeah, I've heard about that too

[thewrongchristian]

Supporting user account when logging into to servers (or when acting as a server yourself) is ok. It's the user account model on the local computer that is outdated. Once I have my ext4 VFS and NTFS VFS I can easily break the security model of both Windows and Linux by booting my own OS on the machine and then being able to access EVERYTHING since I won't support the access control embedded into the filesystem. This is just security by complexity that isn't worth anything.

The classic user account model on local computers has never been designed to protect against somebody who has physical access to the machine (e.g. you can reboot it and boot another OS): it protects users from unintentionally accessing (or writing) data they shouldn't and it allows per-user customization. For example: me and my wife share some home machines. We don't need "security" against each other on the local accounts, but having different accounts is still necessary: each one has his own account with different settings, files and wallpaper. Also, we cannot unintentionally delete each other's files. All of this it's not broken: it's a valid use case, but I agree that nothing to do with security.

Sure, that's valid for a desktop OS. However, Unix forces this model on ANY machine, regardless if it is a desktop OS or an embedded system.

UNIX doesn't force anything on anyone.

The UNIX model, simplistic as it is, is a super user, and not-super user.

So a UNIX based embedded system can quite happily run everything as root, as chances are the embedded system will only be running trusted processes. Or, the embedded system can have a trusted init, which will start sub-processes to do the actual work as whatever unprivileged user id (no accounts are necessary, just use setuid to UID 1 to run unprivileged code).

[rdos]

UNIX doesn't force anything on anyone.

The UNIX model, simplistic as it is, is a super user, and not-super user.

So a UNIX based embedded system can quite happily run everything as root, as chances are the embedded system will only be running trusted processes. Or, the embedded system can have a trusted init, which will start sub-processes to do the actual work as whatever unprivileged user id (no accounts are necessary, just use setuid to UID 1 to run unprivileged code).

I think you still need the root (super user) account, and so you are still forced into the user account model.

[thewrongchristian]

UNIX doesn't force anything on anyone.

The UNIX model, simplistic as it is, is a super user, and not-super user.

So a UNIX based embedded system can quite happily run everything as root, as chances are the embedded system will only be running trusted processes. Or, the embedded system can have a trusted init, which will start sub-processes to do the actual work as whatever unprivileged user id (no accounts are necessary, just use setuid to UID 1 to run unprivileged code).

I think you still need the root (super user) account, and so you are still forced into the user account model.

root (as in, a root user) is simply a user space name for UID 0, the super-user. process 1 (init) is already running as UID 0. Take any UNIX or UNIX like kernel, give it a filesystem with just an init program, and that program can do whatever it wants.

When I log into my computer, the username I use is a user level construct. The user id I'm assigned is a user level construct. At the kernel level, UNIX cares not what my user id is except in regards to distinguishing the super-user (UID 0) from non-super user (any other UID) and any UID from any other UID.

Presumably, an embedded system would not need a user in the desktop sense. It would just make sense to run user code as unprivilegedcode, so the only requirement is to drop UID 0 privileges by setting UID to 1.

[andrew_w]

Regardless of method, it's basically impossible to implement a zero-copy file API with both the shared buffer interface (which is obvious) and with memory mapping. This is because file clusters might be smaller than the page size, which means you cannot memory map the file without allocating an intermediate buffer. What is needed is that the kernel must be able to memory map the file in chunks of it's underlying sector size, and the application must be able to handle that the mapping might not be continous rather in chunks. With traditional memory mapping, it's typically the application that defines where the mapping should reside, which won't work with zero-copy. Unless you create an overly complex disc buffer architecture.

It's true that it is impossible to implement a zero-copy interface with memory mapping for files with a storage-like backing, but it is perfectly possible to implement a zero-copy file interface for files with a memory-like backing (i.e. MMIO or actual RAM) as long as they are page-aligned. UX/RT will directly map any file with a page-aligned memory-like backing into the address space of the target process (although in some cases accesses will be intercepted with page faults). In fact, memory-mapped files will be the only way to map memory at all for regular processes, and anonymous memory will be completely absent outside the kernel and process server.

I'm not building a desktop OS for multiple users, rather I'm more focused on embedded systems and servers/data collectors.

The basic security infrastructure in UX/RT will be pretty lightweight. There will likely be a cut-down permission manager for embedded systems, and it will be possible to run with no permission manager at all. The login manager will be separate from the permission manager and will also be optional.

I suppose you (as the OS creator) could easily add this, but could somebody else do it without changing or adding interfaces to your OS? This is the problem in Windows/Linux. In my design, somebody could actually add a device driver for this that interacts with PCI to find & initiate the device and that exports a set of APIs for user level. This would not require any kernel modifications. The only thing it would require are some private gate numbers that could just be picked at the end.

Yes, it will be easy enough for anyone to do it with stock UX/RT since it will be a QNX-like pure microkernel system. All drivers will be ordinary processes (or plugins loaded into ordinary processes), and the interfaces they use will be available to any process that has been configured with sufficient privileges (and there will be no restrictions on which programs can be given what privileges). There won't be any way to add custom system call traps, but that won't be necessary since literally all interfaces will be based on file-oriented message passing and memory mapping and any process will be able to export a filesystem.

Extensibility is so important to me that I put it in the name (Universally eXtensible Real Time operating system). The poor extensibility of conventional Unices (and all of the ad-hoc hacks needed to work around it) is responsible for many or even most of the problems with them IMO.

I think you still need the root (super user) account, and so you are still forced into the user account model.

root (as in, a root user) is simply a user space name for UID 0, the super-user. process 1 (init) is already running as UID 0. Take any UNIX or UNIX like kernel, give it a filesystem with just an init program, and that program can do whatever it wants.

When I log into my computer, the username I use is a user level construct. The user id I'm assigned is a user level construct. At the kernel level, UNIX cares not what my user id is except in regards to distinguishing the super-user (UID 0) from non-super user (any other UID) and any UID from any other UID.

Presumably, an embedded system would not need a user in the desktop sense. It would just make sense to run user code as unprivilegedcode, so the only requirement is to drop UID 0 privileges by setting UID to 1.

Yes, in Unix-like systems, login management is almost always separated from the actual permission checking, which is generally pretty lightweight. This will be true of UX/RT as well, even though both its login management and permission checking will be rather different from that under legacy Unix.

[rdos]

When I log into my computer, the username I use is a user level construct. The user id I'm assigned is a user level construct. At the kernel level, UNIX cares not what my user id is except in regards to distinguishing the super-user (UID 0) from non-super user (any other UID) and any UID from any other UID.

Well, I don't want to log in. I like it better when I get the command prompt without any fuss.

[bzt]

Well, I don't want to log in. I like it better when I get the command prompt without any fuss.

That's most certainly one way to do it. Single-user OSes do exists (like DOS, Haiku etc.), but they don't provide user based permissions. In a multi-user OS if you don't authenticate the user, then how do you know what's allowed for the user and what's not (or with other words which user is actually using the computer)? User based privileges require some sort of authentication (nickname/password, PIN-code, fingerprint, face-recognition whatever) so that you can know which user's privileges to load when the session starts. It could be that there's only one user allowed to run a session in a multi-user system, and it could be that authentication is a nil, which automatically logs in one of the users.

Simply put: you can't use privileges in a single-user OS, but you can configure a multi-user OS to use only one user. Hope this makes sense to you.

Cheers,
bzt

[rdos]

Well, I don't want to log in. I like it better when I get the command prompt without any fuss.

That's most certainly one way to do it. Single-user OSes do exists (like DOS, Haiku etc.), but they don't provide user based permissions. In a multi-user OS if you don't authenticate the user, then how do you know what's allowed for the user and what's not (or with other words which user is actually using the computer)? User based privileges require some sort of authentication (nickname/password, PIN-code, fingerprint, face-recognition whatever) so that you can know which user's privileges to load when the session starts. It could be that there's only one user allowed to run a session in a multi-user system, and it could be that authentication is a nil, which automatically logs in one of the users.

Simply put: you can't use privileges in a single-user OS, but you can configure a multi-user OS to use only one user. Hope this makes sense to you.

There is a lot to say about this (outdated IMO) method to handle users. Once upon a time, many different tools actually were connected to user accounts, but I think this is getting less common. For instance, telnet was a way to remote login, but practically every server today has telnet disabled. I can setup a telnet server and just let it run the command shell with no login. FTP might be connected to user accounts, but it is not necessary. I don't support the user account model, yet I do support FTP. I have a collection of classes for FTP, and among other things, they also define valid user & password combinations and which root they can access & how (read or read/write). Same thing with HTTP. I have classes for HTTP that can handle authentication (both as server & client). This is part of the C++ classes. How they get the information (fixed settings or files) is up to the programmer.

Another example is SVN. When we installed SVN on our M$ servers, we couldn't connect users to Windows accounts, rather they were defined in a file on the server. Which also adds the advantage that somebody could access SVN without being a user on our internal server. We also use a Windows domain at work, but on one machine I didn't want this, and I concluded I didn't miss anything by not being in the domain. I can easily mount my discs on the server by entering a user & password (that's not on my machine).

Web-applications typically won't use the user account model, and it is the big user base today.

I think that if you have physical access to a machine, then the user account model doesn't provide much of security. After all, I can install my own OS to circumvent lock-downs in the filesystems. If you don't have physical access, then it comes down to protecting the network. I do this by allowing FTP access on my internal network, but not from Internet. On our terminals, this is disabled when they run in production, but active if we "safe boot". This way I can safe boot machines, transfer files and test things easily, while in production nothing can be changed neither with physical access nor from the network. Everything is configured on a central site.

[mid]

I think that if you have physical access to a machine, then the user account model doesn't provide much of security. After all, I can install my own OS to circumvent lock-downs in the filesystems. If you don't have physical access, then it comes down to protecting the network. I do this by allowing FTP access on my internal network, but not from Internet. On our terminals, this is disabled when they run in production, but active if we "safe boot". This way I can safe boot machines, transfer files and test things easily, while in production nothing can be changed neither with physical access nor from the network. Everything is configured on a central site.

This is assuming family isn't in your threat model, or anyone else that is technologically-challenged yet malicious. I wouldn't want my kid ruining the system and deleting all my valuable files, either.

[bzt]

For instance, telnet was a way to remote login, but practically every server today has telnet disabled.

Yeah, because they are using SSH instead. The difference between telnet and SSH is that the latter is using an encrypted tunnel, but this is independent to the user model.

Web-applications typically won't use the user account model, and it is the big user base today.

I don't know what makes you think that. All the big web-applications implement the user account model alright (Facebook, Google, Youtube, TikTok, Twitter, Insta, iCloud, Azure, AWS, etc. etc. etc.) They have replaced HTTP AUTH with a lot more secure, encrypted HTTPS web-form, but you typically have to use a password (sometimes called API-key) to log in to those services. Just because you save your credentials on your phone and use a mobile app instead of a browser, doesn't mean there's no user authentication in the background when accessing the web interface.

All the web services I use are built on user account model too, like github, OSDev forum etc. with typically username/password authentication. Actually when I've tried, I couldn't find any web-application which isn't using the user model...

Cheers,
bzt

[rdos]

Web-applications typically won't use the user account model, and it is the big user base today.

I don't know what makes you think that. All the big web-applications implement the user account model alright (Facebook, Google, Youtube, TikTok, Twitter, Insta, iCloud, Azure, AWS, etc. etc. etc.) They have replaced HTTP AUTH with a lot more secure, encrypted HTTPS web-form, but you typically have to use a password (sometimes called API-key) to log in to those services. Just because you save your credentials on your phone and use a mobile app instead of a browser, doesn't mean there's no user authentication in the background when accessing the web interface.

All the web services I use are built on user account model too, like github, OSDev forum etc. with typically username/password authentication. Actually when I've tried, I couldn't find any web-application which isn't using the user model...

Sure, there is still user authorization but it's not implemented using the user accounts on the local machine. Actually, you don't need to have local user accounts on your server to support authorization with HTTP or HTTPS. While I have not worked on mainstream web apps, I have written a PHP script, and it supports registering users, but this is done by saving the username + MD5 encrypted password in a SQL table, not through creating local user accounts on my virtual server.

[rdos]

I think that if you have physical access to a machine, then the user account model doesn't provide much of security. After all, I can install my own OS to circumvent lock-downs in the filesystems. If you don't have physical access, then it comes down to protecting the network. I do this by allowing FTP access on my internal network, but not from Internet. On our terminals, this is disabled when they run in production, but active if we "safe boot". This way I can safe boot machines, transfer files and test things easily, while in production nothing can be changed neither with physical access nor from the network. Everything is configured on a central site.

This is assuming family isn't in your threat model, or anyone else that is technologically-challenged yet malicious. I wouldn't want my kid ruining the system and deleting all my valuable files, either.

I keep my valuable files in SVN or at my web host.

[bzt]

Sure, there is still user authorization but it's not implemented using the user accounts on the local machine. Actually, you don't need to have local user accounts on your server to support authorization with HTTP or HTTPS. While I have not worked on mainstream web apps, I have written a PHP script, and it supports registering users, but this is done by saving the username + MD5 encrypted password in a SQL table, not through creating local user accounts on my virtual server.

Right, but that's still a user account model. No matter if it's implemented as an OS user or by an application, the model is the same (one account stored for each user which can be accessed using credentials). Just for the records, I did work with web apps, and we did create a separate OS user for each virtual server (so that we could separate them from each other, in case one of our customer's website gets defaced), but not for their users, we left that part to the web-applications running in those virtual servers.
And just an advice, never use MD5 for hashing passwords, it's extremely easy to crack. Use the built-in password_hash (and password_verify) functions instead, as they use much stronger algorithms and random salt to the hash as well.

On my local computer, I still prefer to have user accounts (one for every day use, and one with superuser privileges), to avoid accidental removing of important files, plus I prefer BIOS password too to mitigate evil maid attacks.

Cheers,
bzt