Rewrite from Scratch

Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Rewrite from Scratch

Post by Brendan »

Hi,
Kevin wrote:
Brendan wrote:Different people may have different ideas about where this "point of equality" is. That's fine - every case is different, and it's up to each developer to make up their own mind for their own specific case.

Attempting to argue that this "point of equality" doesn't exist and that incremental change is always better regardless of how much code needs to be redesigned and rewritten make no sense at all. This is what you're trying to do. Your argument/s are obvious nonsense.
No, that's not what I'm doing. I absolutely agree that this break-even point exists. Discussing about its existence would be a waste of time and totally uninteresting. The interesting question is where it is, and I claim that for typical non-trivial projects (let's say > 30 KLOC, just to have a number) you never mess up your design enough that a rewrite from scratch would be the better option - because in practice nobody writes Minecraft when he really wanted to write a kernel.
I started writing my first OS before anyone that I knew (including myself) had internet access. I had no Intel manuals or anyone to explain things or any of that. What I did have was about 20 pages (photo-copied from a book I can't remember) that described 8086 assembly, another book that I "borrowed indefinitely" from a public library that described BIOS functions and not much else (The Peter Norton Programmer's Guide to the IBM PC), and an old assembler (A86). Due to a massive lack of knowledge my first OS was a real mode OS. It could best be described as "single-user, single-tasking, megalithic" (the "OS" was literally one bootable binary containing the OS and its applications).

There wouldn't have been more than about 10 lines of code that was useful for the second version of the OS.

My second OS used protected mode with plain 32-bit paging; and was a "single-user, single-CPU, multi-tasking" micro-kernel (when I started writing it very few people had computers with more than 256 MiB of RAM so things like PAE seemed like a waste of time, and only rare/expensive severs had multiple CPUs so that seemed pointless too). It only supported 80 * 25 text mode. The memory management wasn't too good. The scheduler was crap (hardware task switching, round robin). The device driver model was hideous (device drivers ran at CPL=1 in a special "driver area" that was mapped into every address space). Ironically, the IPC supported both synchronous messaging and asynchronous messaging (I dropped support for synchronous messaging in later kernels). The code itself "worked", but it was limited and messy (poor modularity) and made too many assumptions about the hardware (rock solid on 80486 and Pentium systems, but unreliable on anything newer).

There wouldn't have been more than about 10 lines of code that was useful for the third version of the OS (near then end, every piece of code I looked at looked like it was written by someone that had never written an OS before).

My third OS was similar, but everything was just plain better - support for PAE, support for graphics modes/VBE, much more efficient memory management, greatly improved scheduler, much better driver model, etc (and a lot less stability problems - I'd started to gather a pool of test machines by that stage).

There wouldn't have been more than about 10 lines of code that was useful for the fourth version of the OS.

Times change, and hardware changes with it. Not long after Intel introduced hyper-threading and multi-core CPUs it became obvious that ever-increasing clock speeds was going to stop and be replaced with ever-increasing CPUs instead. At around the same time AMD introduced long mode. My fourth OS was similar to my third in spirit, but very different in implementation - mostly due to multi-CPU and long mode, but also because I was growing more confident and wanted "commercial quality" rather than just "hobbyist" - a greater emphasis on fault tolerance (and error detection and error handling in general), things like CRC checks in executable headers, version numbering to allow "auto-update", my first serious look at internationalisation, etc.

There wouldn't be more than about 10 lines of code that I consider useful for the fifth version of my OS.

When I started writing the fourth version of my OS I just assumed the BIOS would always be there. Along comes UEFI and the design of the boot code (and the interface between the kernel and the boot code) needs some serious redesign. The fourth OS also taught me a lot about locking, scalability and multi-CPU (and just like before, the experience I gained means I can do far better next time). I also started having other ideas - refining the project's goals (resolution independence and colour space independence, hot-plug CPUs, etc). Things that I assumed I'd want to support (e.g. C, HTML, etc) I no longer want. The memory management needs to change, the scheduler needs to change, etc. I don't want my current abilities to be limited by my previous knowledge or past decisions.

I haven't seriously started the fifth version yet. I did a bunch more research (a few boot code prototypes, including messing about with UEFI, etc) and real life got in the way a bit. I'm expecting to start again in a few months.

When you start again from scratch you worry if it's the right thing to do. Even after you've started again it's natural to wonder if you should've kept working away on your old crappy code. It's not until you get about half way through the rewrite and look back at what you've accomplished (and how much better it is than the old code) that you know you made the right decision.

In hindsight, I have no reason to regret any of these rewrites. What I regret most is thinking "it's just a hobby kernel" - if I'd considered it a serious/professional project from the begining maybe I would have spent more time on research and less time on "scope creep".


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Rewrite from Scratch

Post by Antti »

Brendan wrote:[History]
Is this fourth version of the OS publicly available? I checked and noticed that you have a source code viewer for Previous Project on your homepage. It is not very "compiler friendly" but it must be a reasoned decision to do it that way.

I am not very interested in seeing the source code but the actual bootable image would be worth trying.
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Rewrite from Scratch

Post by Brendan »

Hi,
Antti wrote:
Brendan wrote:[History]
Is this fourth version of the OS publicly available? I checked and noticed that you have a source code viewer for Previous Project on your homepage. It is not very "compiler friendly" but it must be a reasoned decision to do it that way.

I am not very interested in seeing the source code but the actual bootable image would be worth trying.
The fourth version isn't publicly available. I went looking for it today and it's not on this computer; and I'm not sure if it survived the transition from the computer I was using when I wrote it (a dual Pentium III server - my first multi-CPU system) to the computer I used after that, to my current development machine.

The "Previous Project" from my web site is just some prototype boot code I wrote (I think it's from the beginning of research into resolution independence) and there never was any kernel/s or anything for it. You can get a bootable image of that from "http://bcos.hopto.org/www2/ftp/".


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Rewrite from Scratch

Post by rdos »

Interesting history, Brendan. We seem to have worked on OSes roughly the same time, but with quite different approaches.

RDOS (the first and only version) was started sometimes in 1988 because I was fed-up with MSDOS inability to multitask. The initial goal was to support multitasking in DOS, and also running multiple copies of the environment. I was lucky to get a copy of the Intel 386 system programming books early on (actually this was at a convent I was attending at work). I found their IA32 model very elegant (and still do), so I naturally based it on that. I lacked C compilers and a linker for RDOS, so I just defined the device-driver modules as DOS executables with no relocations. Since I couldn't link modules, it was natural to develop a code-patching interface that still is in RDOS today. Initially, this was required to link drivers. As DOS used the interrupt model for syscalls, and I regarded that model as inferior and bad, I didn't want to copy it. Especially since DOS switch stacks early in the syscalls, which created all the problems with multitasking (basically, it used an implicit BKL). We might call this the first generation.

The second phase started to use the 16-bit NE format for executables. It also added support for various 16 and 32-bit DOS extenders, and the DPMI interface.

The third generation started to use Win32 console applications. This was done by adding a "virtual executable loader" interface, and adding an PE module. As RDOS already had a multitasking kernel, it was no big deal to provide the Win32 emulation DLLs, and supporting the multithreading interface as well. This progressed to the point where Borlands TD32 debugger (console mode) could be used to debug applications natively.

The forth generation started with porting the application interface to OpenWatcom. This meant I could get rid of the emulation DLLs, and instead I wrote a native library. I also could write a remote debug server so I could debug remotely, which I couldn't do before. I also ported the library for kernel, so I could start to code device-drivers in C, which was not possible before. The assembly-only modules were using their own format as well instead of DOS (actually, there is both a 16-bit and a 32-bit device-driver format today, but C is only supported by the 32-bit format).

The fifth generation removed hardware taskswitching, added SMP support, and all that comes with it like APIC and IO-APIC. This increased compability considerably so now RDOS can run on most modern and older machines. This meant rather large changes to parts of the kernel, while the application and device-driver interface largely remained the same.

I could have started from scratch 5 times, but then I'd probably be in the initial alpha-phase rather than in the stable phase. Somewhere in the second generation I also started to use RDOS as a debug-tool for our terminals. Somewhere in the third generation I developped the standalone terminal, that now is running at a few hundred installations.

I've also done a real-mode kernel, but this was done by porting the scheduler in RDOS to real-mode, not as a separate project. This kernel was used in the terminals that wasn't PC-based and run for many years (and still is).
User avatar
eryjus
Member
Member
Posts: 286
Joined: Fri Oct 21, 2011 9:47 pm
Libera.chat IRC: eryjus
Location: Tustin, CA USA

Re: Rewrite from Scratch

Post by eryjus »

Brendan wrote:I haven't seriously started the fifth version yet. I did a bunch more research (a few boot code prototypes, including messing about with UEFI, etc) and real life got in the way a bit. I'm expecting to start again in a few months.
Have you been working on your design during this time. I would be curious to see the format in which you capture your thoughts for later implementation, if you are willing to share. The components you think about would be a great education for me (and I suspect others).

(Maybe it's time to split this thread...?)
Adam

The name is fitting: Century Hobby OS -- At this rate, it's gonna take me that long!
Read about my mistakes and missteps with this iteration: Journal

"Sometimes things just don't make sense until you figure them out." -- Phil Stahlheber
Kevin
Member
Member
Posts: 1071
Joined: Sun Feb 01, 2009 6:11 am
Location: Germany
Contact:

Re: Rewrite from Scratch

Post by Kevin »

When you start again from scratch you worry if it's the right thing to do. Even after you've started again it's natural to wonder if you should've kept working away on your old crappy code. It's not until you get about half way through the rewrite and look back at what you've accomplished (and how much better it is than the old code) that you know you made the right decision.
Oh yes, I know that feeling when you look back at the old code and you only then realise how much better it's now. But this doesn't require a rewrite from scratch, incremental steps lead to such great changes as well.
In hindsight, I have no reason to regret any of these rewrites. What I regret most is thinking "it's just a hobby kernel" - if I'd considered it a serious/professional project from the begining maybe I would have spent more time on research and less time on "scope creep".
This is funny, because I was going to say that you can probably do all these rewrites for "just a hobby kernel", but you couldn't do it with anything serious that is in production. I think it's the same thing that rdos means when he says "I could have started from scratch 5 times, but then I'd probably be in the initial alpha-phase rather than in the stable phase." And (please excuse me if my impression is wrong there) you're indeed known in the community for being in a permanent inital alpha phase.

Other than that, not sure what you were really intending with your historical overview. I guess from version 1 to 2 I'd have gone for a rewrite as well, but the rest sounds as if it had been at least as sensible to do it incrementally.
Developer of tyndur - community OS of Lowlevel (German)
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Rewrite from Scratch

Post by Brendan »

Hi,
Kevin wrote:
When you start again from scratch you worry if it's the right thing to do. Even after you've started again it's natural to wonder if you should've kept working away on your old crappy code. It's not until you get about half way through the rewrite and look back at what you've accomplished (and how much better it is than the old code) that you know you made the right decision.
Oh yes, I know that feeling when you look back at the old code and you only then realise how much better it's now. But this doesn't require a rewrite from scratch, incremental steps lead to such great changes as well.
Has anyone actually successfully done this? To me it seems like torture - a large amount of "turd polishing" that in theory could produce something that smells like roses, but in practice never does because there's always at least some rotting flesh and weeping scabs left over that never quite make it to the top of the "todo" list.
Kevin wrote:
In hindsight, I have no reason to regret any of these rewrites. What I regret most is thinking "it's just a hobby kernel" - if I'd considered it a serious/professional project from the begining maybe I would have spent more time on research and less time on "scope creep".
This is funny, because I was going to say that you can probably do all these rewrites for "just a hobby kernel", but you couldn't do it with anything serious that is in production. I think it's the same thing that rdos means when he says "I could have started from scratch 5 times, but then I'd probably be in the initial alpha-phase rather than in the stable phase."
The only person here (that I know of) with an OS that is in production is RDOS. However; for an OS that is in production, nothing prevents "rewrite from scratch" anyway - you just put the old/existing version into "maintenance mode", and release the new version when you can, and let people upgrade from old to new in their own time. There's also several cases of companies doing this (e.g. DOS -> Win9x -> WinNT).

For RDOS specifically; embedded systems are very different. The end product is a combination of hardware and software; and once it's been deployed it's almost never updated (nothing you do to the current version of the OS effects previous versions that were deployed in the past, and backward compatibility is mostly irrelevant). I do believe that RDOS would benefit a lot from a rewrite (but I also understand that businesses care more about profit than quality; and the average company would be much more likely to scrap the in-house OS entirely and switch to QNX or something).
Kevin wrote:And (please excuse me if my impression is wrong there) you're indeed known in the community for being in a permanent inital alpha phase.
This is a difference in goals. Most people here only want to learn and aren't attempting to surpass existing OSs or do anything revolutionary. For this case "good enough" is more than adequate. The question is, what comes after "writing an OS for learning/fun"?

For me, what comes next is attempting to create something that does surpass existing OSs. For this "good enough" isn't even close to adequate. An OS is like a hierarchy of abstractions (e.g. boot code, with kernel/s on top of that, with drivers on top of that, with things like VFS and GUI on top of that, with applications and things on top of that). If one piece is only "good enough" (not even close to adequate) then it's far better to replace it with something that actually is adequate before too much depends on it (or to put it another way; you don't build a sky-scraper on sand). Until I reach the point where I'm unable to think of a way to improve the design of the boot code, I refuse to bother with the kernel.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Rewrite from Scratch

Post by rdos »

Brendan wrote:For RDOS specifically; embedded systems are very different. The end product is a combination of hardware and software; and once it's been deployed it's almost never updated (nothing you do to the current version of the OS effects previous versions that were deployed in the past, and backward compatibility is mostly irrelevant). I do believe that RDOS would benefit a lot from a rewrite (but I also understand that businesses care more about profit than quality; and the average company would be much more likely to scrap the in-house OS entirely and switch to QNX or something).
RDOS is not really in-house. It is my project, but I let my company use it for free. I can let other companies use it as well, and charge a fee for that.

My goal is that I will never, ever, have to code in Windows or Linux again, and I think I have some possibility to achieve it. Most of the competitors use Windows XP embedded for similar projects, but our solution is much easier to deploy than Microsofts, and can be upgraded over GPRS without much problems, while Windows XP cannot.
Kevin
Member
Member
Posts: 1071
Joined: Sun Feb 01, 2009 6:11 am
Location: Germany
Contact:

Re: Rewrite from Scratch

Post by Kevin »

Brendan wrote:
Kevin wrote:Oh yes, I know that feeling when you look back at the old code and you only then realise how much better it's now. But this doesn't require a rewrite from scratch, incremental steps lead to such great changes as well.
Has anyone actually successfully done this?
Yes, that's common practice in many projects. Just to name one well-known project that even develops a kernel: Ever heard of Linux?

Kevin wrote:This is funny, because I was going to say that you can probably do all these rewrites for "just a hobby kernel", but you couldn't do it with anything serious that is in production. I think it's the same thing that rdos means when he says "I could have started from scratch 5 times, but then I'd probably be in the initial alpha-phase rather than in the stable phase."
The only person here (that I know of) with an OS that is in production is RDOS.
Quite possibly (I won't count my few lines in the Linux kernel). But it doesn't matter, you said you'd consider your kernel a "serious/professional project", which I think definitely includes maintaining stability as if it were in production.
However; for an OS that is in production, nothing prevents "rewrite from scratch" anyway - you just put the old/existing version into "maintenance mode", and release the new version when you can, and let people upgrade from old to new in their own time. There's also several cases of companies doing this (e.g. DOS -> Win9x -> WinNT).
I think you're aware how much work Microsoft had to put into the compatibility layers each time they did this.
For me, what comes next is attempting to create something that does surpass existing OSs. For this "good enough" isn't even close to adequate. An OS is like a hierarchy of abstractions (e.g. boot code, with kernel/s on top of that, with drivers on top of that, with things like VFS and GUI on top of that, with applications and things on top of that). If one piece is only "good enough" (not even close to adequate) then it's far better to replace it with something that actually is adequate before too much depends on it (or to put it another way; you don't build a sky-scraper on sand). Until I reach the point where I'm unable to think of a way to improve the design of the boot code, I refuse to bother with the kernel.
As you write your kernel only as a hobby, you can choose to do it this way. You'll never get a working OS this way, because there's always something to improve in each layer, but there's no requirement that a hobby project ever creates a final product. So, yes, that may be a fine way for you. It probably doesn't work quite as well if you want to get a result in the forseeable future.
Developer of tyndur - community OS of Lowlevel (German)
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Rewrite from Scratch

Post by Antti »

Kevin wrote:there's no requirement that a hobby project ever creates a final product
I like the idea of making the "perfect system" that could not be improved further. However, like you said, it takes infinite amount of time to do that. I do not know what the intention is but the final product would be nice to see within a few years. If it takes a lot longer, the end result might not be very current when released.

It is also interesting how "serious/professional" is defined here in this context. Is it just the quality of end result (software) that counts? Does it include work methods, project management, schedule, documentation, support, etc.

When it comes to hobby projects, all the boring* serious/professional project requirements can usually be ignored. It might be actually easier to focus on making the "perfect system" if it were a hobby project.


*(Highly debatable notion)
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Rewrite from Scratch

Post by Brendan »

Hi,
Kevin wrote:
Brendan wrote:
Kevin wrote:Oh yes, I know that feeling when you look back at the old code and you only then realise how much better it's now. But this doesn't require a rewrite from scratch, incremental steps lead to such great changes as well.
Has anyone actually successfully done this? To me it seems like torture - a large amount of "turd polishing" that in theory could produce something that smells like roses, but in practice never does because there's always at least some rotting flesh and weeping scabs left over that never quite make it to the top of the "todo" list.
Yes, that's common practice in many projects. Just to name one well-known project that even develops a kernel: Ever heard of Linux?
You can't be serious. Linux is like an airship made out of solid concrete that doesn't crash to the ground because it's sitting on top of a huge pile of crushed bodies.
Kevin wrote:
Kevin wrote:This is funny, because I was going to say that you can probably do all these rewrites for "just a hobby kernel", but you couldn't do it with anything serious that is in production. I think it's the same thing that rdos means when he says "I could have started from scratch 5 times, but then I'd probably be in the initial alpha-phase rather than in the stable phase."
The only person here (that I know of) with an OS that is in production is RDOS.
Quite possibly (I won't count my few lines in the Linux kernel). But it doesn't matter, you said you'd consider your kernel a "serious/professional project", which I think definitely includes maintaining stability as if it were in production.
I'm not sure what you mean by "maintaining stability". If you mean making sure public APIs, etc are stable (and I don't keep breaking everything because of API changes after release), then it's important to get rid of any stupid mistakes I've made in the past that are likely to cause API breakage after the first release.

If you mean making sure the code doesn't crash, then it's important to discard all the old bugs I don't know about and write new code (with a much lower chance of hidden bugs); and (while doing this) also add much stronger fault detection and fault tolerance (to guard against the risk of stability problems caused by hardware failure).

I'm not sure how this could be hard to understand - most people get better with experience. Maybe you're like a programmer's version of the Curious Case of Benjamin Button; where you didn't create any bugs as a beginner with no experience at all, but as time passes your code quality deteriorates until you end up as an old man that can't write more than 4 lines of code without screwing something up...
Kevin wrote:
However; for an OS that is in production, nothing prevents "rewrite from scratch" anyway - you just put the old/existing version into "maintenance mode", and release the new version when you can, and let people upgrade from old to new in their own time. There's also several cases of companies doing this (e.g. DOS -> Win9x -> WinNT).
I think you're aware how much work Microsoft had to put into the compatibility layers each time they did this.
And yet despite doing it before (DOS -> Win9x) and despite knowing how much work it'd be they chose to do it again (Win9x - > WinNT); and each time they've done it they've ended up with a far better product than they had before the rewrite.

Now compare "far better product than they had before the rewrite" to "my 32-bit protected mode kernel grew larger than 64 KiB, so I had to shift the scheduler out of the kernel".
Kevin wrote:
For me, what comes next is attempting to create something that does surpass existing OSs. For this "good enough" isn't even close to adequate. An OS is like a hierarchy of abstractions (e.g. boot code, with kernel/s on top of that, with drivers on top of that, with things like VFS and GUI on top of that, with applications and things on top of that). If one piece is only "good enough" (not even close to adequate) then it's far better to replace it with something that actually is adequate before too much depends on it (or to put it another way; you don't build a sky-scraper on sand). Until I reach the point where I'm unable to think of a way to improve the design of the boot code, I refuse to bother with the kernel.
As you write your kernel only as a hobby, you can choose to do it this way. You'll never get a working OS this way, because there's always something to improve in each layer, but there's no requirement that a hobby project ever creates a final product. So, yes, that may be a fine way for you. It probably doesn't work quite as well if you want to get a result in the forseeable future.
For an OS; people aren't going to consider changing from whatever they're currently using to a new OS if the new OS is only slightly better for their purpose (there's a significant amount of "momentum" to overcome, and it's not just retraining/learning and replacing applications). The new OS has to be significantly better than whatever they currently use (for their specific purpose) to overcome this momentum. Releasing something that fails to overcome this momentum is a waste of time - the only effect it has is to give people bad first impressions.

Of course I'm not saying that I will succeed, or that it's likely that people will ever use my OS as more than a toy. I am saying that I'm planning an attempt at success though (regardless of how unlikely success is).


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Rewrite from Scratch

Post by Brendan »

Hi,
Antti wrote:
Kevin wrote:there's no requirement that a hobby project ever creates a final product
I like the idea of making the "perfect system" that could not be improved further. However, like you said, it takes infinite amount of time to do that. I do not know what the intention is but the final product would be nice to see within a few years. If it takes a lot longer, the end result might not be very current when released.
Please note that I'm not saying it'd take an infinite amount of time. I'm estimating about 3 years for the kernel/s, simple device drivers, simple GUI and a few utilities/applications (mostly enough to get APIs/framework to build on); then another 5 years for the language/toolchain/IDE, followed by another 7 years for more/better drivers, GUI improvements, more applications, etc. After that I'll be ready to release the first public version, which won't be a complete/working system but will hopefully be enough for people to be amazed by its potential (and attract volunteers/developers and/or funding).
Antti wrote:It is also interesting how "serious/professional" is defined here in this context. Is it just the quality of end result (software) that counts? Does it include work methods, project management, schedule, documentation, support, etc.
For me, it's really about "intent" - e.g. if you intend for the OS to become a usable product that millions of people will use daily (regardless of whether anyone ever does use it). Everything else (quality of design, quality of implementation, documentation, regression testing, "polish", etc) is just a side effect of the original intent.
Antti wrote:When it comes to hobby projects, all the boring* serious/professional project requirements can usually be ignored. It might be actually easier to focus on making the "perfect system" if it were a hobby project.
It'd be easier to focus on making a prototype as a hobby project. Unless the intention is there, people (including me) tend not to consider things that need to be taken into account for a "professional" project.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Rewrite from Scratch

Post by Antti »

Brendan wrote:[Schedule]
Three years for the kernel/s and basics does not sound very bad at all. It is quite plausible. Is the kernel meant to be highly portable? I am just questioning the long gap (5 + 7 years) before the first public release. Of course there must be non-public releases for testing purposes? If it took so long to release the system publicly, I think the kernel should be considered being "just an extremely good design and stable API." I am not sure whether the kernel implementation details are very important when looking at the whole big picture. Of course it is quite difficult to forecast the future. It may be that the current hardware is still current in the future.

The language/toolchain/IDE would probably be the most interesting part of the project. It may be the way to do the next big thing. It could "live forever" if it were successful. The hardware evolution does not affect to it.
Brendan wrote:For me, it's really about "intent" - e.g. if you intend for the OS to become a usable product that millions of people will use daily (regardless of whether anyone ever does use it).
That is the right attitude. These kind of projects are interesting to watch. In contrast to "just a hobby project that is discouraged (by the author) not to (even accidentally) gain attention."
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Rewrite from Scratch

Post by Brendan »

Hi,
Antti wrote:
Brendan wrote:[Schedule]
Three years for the kernel/s and basics does not sound very bad at all. It is quite plausible. Is the kernel meant to be highly portable?
My kernels aren't meant to be portable. There will be one designed specifically for each target (one for 32-bit 80x86, one for 64-bit 80x86, etc). This is because I use a lot of target-specific tricks, and will use assembly to write the kernels (as I want to be able to build the OS using a native tools later, where "native tools" includes an assembler but will not include a C compiler). However, I should point out that this is a micro-kernel (physical and virtual memory management, scheduler, IPC/messaging code, and not too much else), and that I'm not going to worry about other CPUs (e.g. ARM) until the 80x86 version of the OS can be used for development.
Antti wrote:I am just questioning the long gap (5 + 7 years) before the first public release. Of course there must be non-public releases for testing purposes?
There'd probably be a few non-public releases (e.g. on these forums) and people would be able to download the latest development snapshot whenever they like. I just won't be announcing it to the wider public (e.g. on sites like osnews.com, etc). If you remember the first public release of KDE4 you'll understand why releasing too early can be a mistake... :)
Antti wrote:If it took so long to release the system publicly, I think the kernel should be considered being "just an extremely good design and stable API." I am not sure whether the kernel implementation details are very important when looking at the whole big picture. Of course it is quite difficult to forecast the future. It may be that the current hardware is still current in the future.
For 80x86, I'm not really expecting any major changes that effect my kernels in the next 10 years (nothing as radical as the increase in multi-core, the introduction of long mode or the change to UEFI). I am expecting minor changes; like a slightly different variation of long mode paging (with larger virtual address space size), wider SIMD/AVX and more CPUs (and more NUMA, given that x2APIC is limited to 16 CPUs per NUMA domain). There's also a few other changes coming (e.g. hardware support for transactional memory) that probably won't effect my design at all. For unexpected changes I'll have to cope if/when the time comes.
Antti wrote:The language/toolchain/IDE would probably be the most interesting part of the project. It may be the way to do the next big thing. It could "live forever" if it were successful. The hardware evolution does not affect to it.
Hardware evolution (e.g. the recent increase in mobile computing) can effect user interface design. It's something I'm aware of though, which means I can take it into account during the initial design and avoid trying to force people to adapt to a different user interface later.
Antti wrote:
Brendan wrote:For me, it's really about "intent" - e.g. if you intend for the OS to become a usable product that millions of people will use daily (regardless of whether anyone ever does use it).
That is the right attitude. These kind of projects are interesting to watch. In contrast to "just a hobby project that is discouraged (by the author) not to (even accidentally) gain attention."
:)


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Rewrite from Scratch

Post by Antti »

Last few days have been terrible. I started rewriting the system from scratch. There is not much code yet. Mainly just stubs that are used for testing the build system. At first there are two targets: x86-32 and x86-64. I try to do things so that it would be easy to add more targets in the future. Portability and simplicity are design goals. Efficiency is not the primary concern.

I will set up a website for this project. I want to own all the rights to the system but I prefer to show the code. Are there any legal problems if I just put "Copyright [...] All Rights Reserved." and just show the code? Is it still proprietary so that I can do whatever I want with it? It goes without saying that I do not use any GPL (or similar) code in it.
Post Reply