Future of Operating Systems

[cxzuk]

Perhaps we finally get "universal" interface at hardware level? To add more, perhaps the functionality mechanism is available on firmware? Then the actual OS just takes care of policy.

In order to integrate an external component into your (operating) system, you will have to define the protocol(/interface) somewhere. And policies are then kept inside some kind of controller. I could imagine this becoming hardware based instead of software based, which i think microsoft has already explored?

As an historical side note: when GNU* project started, they made user space programs first. After then, it was time for kernel. For example, there are some members here who have a high goal of making their operating system to success. However, they do this in reverse order. Although, here in this forum we concentrate more on kernel so I cannot know their priorities for sure. It does not matter in which order to make things if everything is done perfectly. However, my current opinion is that user space things have more important role in achieving the ambitious goal.

I get the impression this is still true for the majority; A lot of people want to port over existing user space applications.

[Antti]

A lot of people want to port over existing user space applications.

Yes, that is what I want too. I am not capable of designing a programming language, programming model, ecosystem, etc. The whole thing that would be needed if doing something revolutionary. "New wheel" instead of reinventing the old wheel. I am stuck in C/Unix way of thinking and best I can do is just to do something that has been done already. My OS is just an implementation and small differences do not change that fact when looking at the whole big picture.

OK, I am not even capable of thinking so complicated things. I will just continue implementing my ELF parser for a boot loader...

[Brendan]

Hi,

As an historical side note: when GNU* project started, they made user space programs first. After then, it was time for kernel. For example, there are some members here who have a high goal of making their operating system to success. However, they do this in reverse order. Although, here in this forum we concentrate more on kernel so I cannot know their priorities for sure. It does not matter in which order to make things if everything is done perfectly. However, my current opinion is that user space things have more important role in achieving the ambitious goal.

If you're not making a clone, it does matter which order you make things - can be very difficult to begin with user-space first.

For an example, if I wanted to write an application for my OS that displayed a "What is your name" dialog box and then read a file to see if the name the user entered is in that file, but I wanted to implement it on Linux first and port it to my OS later; then I'd have to:

• Write a toolchain for Linux that's I can use as a cross-compiler (to compile applications written in my programming language) - there's no point writing applications in a language like C when I have no intention of allowing applications written in C to be ported to my OS.
• Write code to simulate my kernel API, including:
  • code to simulate my OS's virtual memory management, including thread specific storage (that is used for message buffers, etc)
  • code that uses something like datagram sockets internally to simulate the behaviour of my "everything is asynchronous messaging" IPC
  • signal handlers to either hide signals or convert them into messages
  • code to simulate my OS's "spawnProcess()" (possibly just a wrapper around fork/exec); so that the application can spawn a "text box widget" process
  • code to simulate my OS's "spawnThread()" (possibly just a wrapper around pthreads)
• Write something to support my OS's "user interface protocol", so that the application can open a connection and send messages describing what to draw and receive messages when a key is pressed or a mouse button is clicked. For this I'd probably start with SDL, OpenGL and FreeType. This would probably need to be a separate process.
• Figure out how to implement "obituaries". If one of the processes dies or exits, then the OS is meant to send messages to inform other processes (e.g. so that the GUI can close any "user interface protocol" connections, the "text box widget" can terminate itself if its parent process dies, etc).
• Write code to simulate my VFS. This is like an FTP server - something that responds to requests (open, read, write, close and directory lookup messages). It would need to convert my OS's file names and paths into something that will work on Linux, including support for file versioning (e.g. so the application can ask to open the most recent version of the file "foo/bar" or can ask for a specific version of the file "foo/bar").
• Write code to support automatic file conversion. For example, if the application asks to open the file "foo/bar" as UTF32 text but the file happens to be UTF8 text, then the VFS might spawn a process to convert UTF8 text into UTF32 format without the application itself knowing. For Linux, this would also allow my application (which would be expecting a native file format) to read "plain text" files (that are not in my native file format). Of course on Linux the file system doesn't store any useful metadata so I'd have to use guesswork to figure out what format any files are in.

Once all this is done; the application itself and the "text box widget" it relies on could be ported to my OS later; and all the rest is about 3 years of wasted effort.

Writing a clone of an existing OS (e.g. a *nix clone) would be a lot easier; but (if you're doing an OS for more than just your own personal fun/education) nobody wants "yet another *nix clone" (mostly because there's already several good free *nix clones that the people who do want *nix can choose from) so the idea of porting a bunch of *nix/GNU software to your OS is just an easy way to successfully fail.


Cheers,

Brendan

[rdos]

Writing a clone of an existing OS (e.g. a *nix clone) would be a lot easier; but (if you're doing an OS for more than just your own personal fun/education) nobody wants "yet another *nix clone" (mostly because there's already several good free *nix clones that the people who do want *nix can choose from) so the idea of porting a bunch of *nix/GNU software to your OS is just an easy way to successfully fail.

The other way around is also a successful fail. If you attempt to make your OS so different that no existing applications will run, you make sure that your OS neither has useful drivers nor anything useful to run, which is the ultimate failure.

It is not necessary to write an *nix clone in order to be able to use GNU software. Cygwin runs GNU software on top of Windows, and Windows is nothing like *nix. The win-win situation is if the target OS is adaptable to a wide range of different software platforms so it is easy to write interfaces for popular software adapted to POSIX or Windows. By doing something radically different you only make sure that these adaptations will fail, and that nobody would want to use your OS. The way to do it is to write your own usermode-adaptations, not to design an API that cannot be adapted to various uses. Things like automatic file conversions belong in this layer. IPC is an hidden layer in the OS that should not be seen in the usermode API. An application shouldn't care about how the kernel does its job.

[Brendan]

Hi,

Writing a clone of an existing OS (e.g. a *nix clone) would be a lot easier; but (if you're doing an OS for more than just your own personal fun/education) nobody wants "yet another *nix clone" (mostly because there's already several good free *nix clones that the people who do want *nix can choose from) so the idea of porting a bunch of *nix/GNU software to your OS is just an easy way to successfully fail.

The other way around is also a successful fail. If you attempt to make your OS so different that no existing applications will run, you make sure that your OS neither has useful drivers nor anything useful to run, which is the ultimate failure.

If your OS is interesting, then you can write your own apps until other people find it interesting enough to write more software. You might call this "successfully failing temporarily". If your OS is not interesting then nobody will ever bother writing software specifically for your OS (they'll just churn out the normal "generic *nix" projects and port them), and your OS will probably never offer anything better than all the other *nix clones. You might call this "successfully failing permanently".

It is not necessary to write an *nix clone in order to be able to use GNU software. Cygwin runs GNU software on top of Windows, and Windows is nothing like *nix. The win-win situation is if the target OS is adaptable to a wide range of different software platforms so it is easy to write interfaces for popular software adapted to POSIX or Windows. By doing something radically different you only make sure that these adaptations will fail, and that nobody would want to use your OS. The way to do it is to write your own usermode-adaptations, not to design an API that cannot be adapted to various uses. Things like automatic file conversions belong in this layer. IPC is an hidden layer in the OS that should not be seen in the usermode API. An application shouldn't care about how the kernel does its job.

Writing an OS that is different/interesting, and then slapping on some sort of compatibility layer for other OSs software is fine (or at least it's much better than not writing an OS that is different/interesting). The risk here is that it's far too easy for the OS to be overwhelmed by the other OS's software and end up as an "accidental clone". To avoid this you need enough native applications (that are designed to make good use of whatever makes the OS interesting/unique). For example, Windows had plenty of applications before Cygwin came along and Windows hasn't become a boring *nix clone; and Linux had plenty of *nix applications before Wine came along and Linux hasn't become a boring Windows clone; but despite having several different/interesting features Minix 3 never had any applications that make good use of those features and is a boring *nix clone that will never be better than Linux and free/open/netBSD.

Basically; if you want people to use your OS instead of another OS then you have to give them a reason to use your OS instead of another OS. Something like "this OS can run GCC and Apache and GIMP and Firefox!" is not a reason for people to use your OS instead of another OS.


Cheers,

Brendan

[Antti]

fail

failing

fail

failure

If an OS is excellent, different and interesting but does not get any success, it has failed on getting success. It does not mean it would be bad as a product.

If we had an ideal kernel for Unix-like systems, I do not think it would replace current mainstream kernels. In a long run, maybe. "New user-space order" is what we need. I agree with Brendan that native programs are very important. If you had a good set of reference programs on your OS that use OS specific features and are programmed in your modern programming language and follow your OS recommendations of programming, you could get third party programs that also follow the same "quality standard." That would lead to a nice and elegant user-space environment.

Now the open-source community have a free mainstream OS, Linux. Do you think we could made any radical changes to it? It was Unix-like, it is Unix-like and it will be Unix-like. I mean that we are really stuck to it. Without total "rewrite from scratch" things do not change very much. It is another question whether they should change or not.

[rdos]

Writing an OS that is different/interesting, and then slapping on some sort of compatibility layer for other OSs software is fine (or at least it's much better than not writing an OS that is different/interesting). The risk here is that it's far too easy for the OS to be overwhelmed by the other OS's software and end up as an "accidental clone".

Absolutely. The ideal approach is to know about other APIs while still designing things the way you want, but remembering that somehow common APIs should be possible to emulate on top of it, possibly with reduced functionality, but still run.

To avoid this you need enough native applications (that are designed to make good use of whatever makes the OS interesting/unique). For example, Windows had plenty of applications before Cygwin came along and Windows hasn't become a boring *nix clone; and Linux had plenty of *nix applications before Wine came along and Linux hasn't become a boring Windows clone; but despite having several different/interesting features Minix 3 never had any applications that make good use of those features and is a boring *nix clone that will never be better than Linux and free/open/netBSD.

Basically; if you want people to use your OS instead of another OS then you have to give them a reason to use your OS instead of another OS. Something like "this OS can run GCC and Apache and GIMP and Firefox!" is not a reason for people to use your OS instead of another OS.

That's not the way I see it at my current stage. It is a problem that I cannot run Java-applications in RDOS, because it requires an additional UCM controller module running Linux. A user might also want to use PHP or MySQL, and I think it is a bad idea to write my own database-engine.

And if my API cannot somehow emulate POSIX, then porting Java, PHP and MySQL becomes troublesome, and these free projects might not accept extensive patches to their mainline for an unknown OS, meaning you must keep and update patches manually yourself.

[cxzuk]

Linux. Do you think we could made any radical changes to it? It was Unix-like, it is Unix-like and it will be Unix-like. I mean that we are really stuck to it. Without total "rewrite from scratch" things do not change very much. It is another question whether they should change or not.

I think its good to ask yourself if its worth while or if theres a need. I personally feel there is. As things have become more complex, Theres been a push for clarity and maintainability. I dont think we have that (in our languages, tools, or operating systems/kernels) yet.