Rewrite from Scratch

[Griwes]

Personally, I don't want to write the same code twice, which is why I never want to start from scratch.

We know, you've stated it in this thread several times.

[Brendan]

Hi,

I think rewriting OS is good practice at least from my point of view. I did it many times but always because of design changes.

That is self-contradictory. It means your design was bad. This is the very reason for which so many software design books exist.

Not necessarily. There are at least 3 causes of design change:

• bad initial design - e.g. the design wasn't very good for "foo" to begin with. Everyone makes bad design decisions sooner or later (the only people who haven't made bad design decisions are people that haven't made design decisions).
• internal scope creep - e.g. the design was perfectly fine for "foo" which is all we cared about at the time; but now we want to do "bar" and the design needs to change. A simple example of this would be saying "it's only a small hobby OS, therefore I don't want to bother with Unicode and proportional fonts" and then 3 years later thinking "it's grown beyond just a small hobby OS, therefore I actually do want Unicode and proportional fonts".
• external scope creep - e.g. the design was perfectly fine for "foo" which is all we could have known about at the time; but now someone else has introduced "bar" and the design needs to change. At the moment there's probably a lot of people looking at their "it was reasonable to assume PC BIOS at the time" boot code.

Cheers,

Brendan

[rdos]

internal scope creep - e.g. the design was perfectly fine for "foo" which is all we cared about at the time; but now we want to do "bar" and the design needs to change. A simple example of this would be saying "it's only a small hobby OS, therefore I don't want to bother with Unicode and proportional fonts" and then 3 years later thinking "it's grown beyond just a small hobby OS, therefore I actually do want Unicode and proportional fonts".

I don't see that as a valid reason to start from scratch. I went through the process to convert from "custom" 8-bit fonts to TrueType Unicode only half a year ago. I decided against wide-character codings (because that tends to pollute all character-based code in a system) and decided to use UTF-8 throughout. I even actively disabled wide-character support in OpenWatcom's RDOS libc because I didn't want to have anything to do with wide character sets.

[Brendan]

Hi,

internal scope creep - e.g. the design was perfectly fine for "foo" which is all we cared about at the time; but now we want to do "bar" and the design needs to change. A simple example of this would be saying "it's only a small hobby OS, therefore I don't want to bother with Unicode and proportional fonts" and then 3 years later thinking "it's grown beyond just a small hobby OS, therefore I actually do want Unicode and proportional fonts".

I don't see that as a valid reason to start from scratch.

And I didn't say that that alone was a valid reason to start from scratch. I was replying to Love4Boobies' "design changes mean bad design" comment.

I went through the process to convert from "custom" 8-bit fonts to TrueType Unicode only half a year ago. I decided against wide-character codings (because that tends to pollute all character-based code in a system) and decided to use UTF-8 throughout. I even actively disabled wide-character support in OpenWatcom's RDOS libc because I didn't want to have anything to do with wide character sets.

Then Love4Boobies is saying that your original (ASCII, 8-bit fonts?) design was bad. Do you agree with him? I'm saying that originally it may have been good design, and "internal scope creep" happened, causing it to need redesigning. For simplicity; I'll just assume you agree with me.

Where we don't agree is that I think these smaller design changes can become a larger collection that does justify a rewrite; while you think that (for an example) it would've made more sense for Microsoft to keep polishing "MS-DOS" until it becomes something like Windows 8 without starting again (twice).


Cheers,

Brendan

[Love4Boobies]

• bad initial design - e.g. the design wasn't very good for "foo" to begin with. Everyone makes bad design decisions sooner or later (the only people who haven't made bad design decisions are people that haven't made design decisions).

I don't see how the fact that everyone makes mistakes is an argument for the fact that requiring a rewrite is not a sign of a good design.

• internal scope creep - e.g. the design was perfectly fine for "foo" which is all we cared about at the time; but now we want to do "bar" and the design needs to change. A simple example of this would be saying "it's only a small hobby OS, therefore I don't want to bother with Unicode and proportional fonts" and then 3 years later thinking "it's grown beyond just a small hobby OS, therefore I actually do want Unicode and proportional fonts".
• external scope creep - e.g. the design was perfectly fine for "foo" which is all we could have known about at the time; but now someone else has introduced "bar" and the design needs to change. At the moment there's probably a lot of people looking at their "it was reasonable to assume PC BIOS at the time" boot code.

A good design would have to be flexible enough to allow for the requirements to change over time.

Then Love4Boobies is saying that your original (ASCII, 8-bit fonts?) design was bad. Do you agree with him? I'm saying that originally it may have been good design, and "internal scope creep" happened, causing it to need redesigning. For simplicity; I'll just assume you agree with me.

Using such a font is not a sign of bad design. Inability to switch to vector fonts because of the design on the other hand...

Where we don't agree is that I think these smaller design changes can become a larger collection that does justify a rewrite; while you think that (for an example) it would've made more sense for Microsoft to keep polishing "MS-DOS" until it becomes something like Windows 8 without starting again (twice).

You are starting from the assumption that their designs were good when they actually weren't. They may have seemed good at the time; that's another story.

Also, notice that my main problem with that post was that the poster was advocating rewrites as a good practice.

[rdos]

Where we don't agree is that I think these smaller design changes can become a larger collection that does justify a rewrite; while you think that (for an example) it would've made more sense for Microsoft to keep polishing "MS-DOS" until it becomes something like Windows 8 without starting again (twice).

MS-DOS is the worse piece of junk ever launched, so I don't think Microsoft should have polished on it. They should have released a multitasking system to start with, not such a piece of junk as they did. MS-DOS was the primary reason why I went into OS development. My initial design goal was to write an OS that was compatible with MS-DOS, with multithreading and multiprocess support (the "r" in RDOS stands for "realtime DOS"). Because it was already in the late 80s evident that running programs in the limited real mode environment was a dead-end approach, I designed the new OS for running in 32-bit protected mode, and running DOS applications in V86 mode. Because I planned to run multiple DOS programs at the same time, it was an requirement to use paging. Segmentation was used because it was such an elegant solution for within and between modules isolation, enforced by hardware.

[rdos]

• bad initial design - e.g. the design wasn't very good for "foo" to begin with. Everyone makes bad design decisions sooner or later (the only people who haven't made bad design decisions are people that haven't made design decisions).

I don't see how the fact that everyone makes mistakes is an argument for the fact that requiring a rewrite is not a sign of a good design.

Using raster fonts at the time this was implemented was considered an acceptable design. I'm not even sure if TrueType fonts were available at that time, but at least there were no free code to use.

• internal scope creep - e.g. the design was perfectly fine for "foo" which is all we cared about at the time; but now we want to do "bar" and the design needs to change. A simple example of this would be saying "it's only a small hobby OS, therefore I don't want to bother with Unicode and proportional fonts" and then 3 years later thinking "it's grown beyond just a small hobby OS, therefore I actually do want Unicode and proportional fonts".
• external scope creep - e.g. the design was perfectly fine for "foo" which is all we could have known about at the time; but now someone else has introduced "bar" and the design needs to change. At the moment there's probably a lot of people looking at their "it was reasonable to assume PC BIOS at the time" boot code.

A good design would have to be flexible enough to allow for the requirements to change over time.

Exactly. I had no big problems changing from GFT-based raster fonts to OpenType fonts. I simply rewrote the font device-driver from scratch in C, and integrated it into the available framework.

[Love4Boobies]

@rdos: Saying that MS-DOS was the biggest piece of junk is going overboard. Back then, computers were quite slow, expensive, had rudimentary graphics hardware, and there was no protected mode. There was also limited use for them, Microsoft didn't have many resources, etc. An OS like the one you descrined could not have existed for the PC back then.

Besides, it's very easy to point out bad design in hindsight but history teaches us that people have been consistently bad at predicting the future. Not only that, but good ideas only seem natural once they're out there. Your grandchildren will probably be amused that your generation never thought of FOOBAR---after all, it was so simple!

[rdos]

@rdos: Saying that MS-DOS was the biggest piece of junk is going overboard. Back then, computers were quite slow, expensive, had rudimentary graphics hardware, and there was no protected mode. There was also limited use for them, Microsoft didn't have many resources, etc. An OS like the one you descrined could not have existed for the PC back then.

But there were a multitasking system back then called CP/M (look here: http://en.wikipedia.org/wiki/CP/M). CP/M was a decent operating system, but for some reason IBM let Bill Gates do his hack and used that instead. If IBM had used CP/M instead, the PC market would be very different from how it is now, and it wouldn't have taken two major rewrites by Microsoft until there was decent multitasking on PCs.

And while the 386 processor didn't exist initially when MS-DOS was written, it was on the market 1988 when I started my project. And I bet the 386 processor was at least at the early design-stages when MS-DOS was written.

[Brendan]

Hi,

• bad initial design - e.g. the design wasn't very good for "foo" to begin with. Everyone makes bad design decisions sooner or later (the only people who haven't made bad design decisions are people that haven't made design decisions).

I don't see how the fact that everyone makes mistakes is an argument for the fact that requiring a rewrite is not a sign of a good design.

Imagine someone who's never designed or implemented an OS before. Is it sane to expect them to design and implement something perfect on their first attempt?

Excluding "clones" (where an existing OS design is merely copied and no design flaws are possible) I can't think of anything that's actually good which hasn't been through at least 2 rewrites.

• internal scope creep - e.g. the design was perfectly fine for "foo" which is all we cared about at the time; but now we want to do "bar" and the design needs to change. A simple example of this would be saying "it's only a small hobby OS, therefore I don't want to bother with Unicode and proportional fonts" and then 3 years later thinking "it's grown beyond just a small hobby OS, therefore I actually do want Unicode and proportional fonts".
• external scope creep - e.g. the design was perfectly fine for "foo" which is all we could have known about at the time; but now someone else has introduced "bar" and the design needs to change. At the moment there's probably a lot of people looking at their "it was reasonable to assume PC BIOS at the time" boot code.

A good design would have to be flexible enough to allow for the requirements to change over time.

Sounds like a nice theory. Can you think of anything that has actually achieved this in practice? If both Microsoft and Apple can't do it, should we expect random hobbyists to be able to?

In practice, this argument is like saying that the person who designed the first horse-drawn cart was a fool because he should've predicted the invention of combustion engines and designed the horse-drawn cart to easily adapt to unforeseeable changes in technology.

Where we don't agree is that I think these smaller design changes can become a larger collection that does justify a rewrite; while you think that (for an example) it would've made more sense for Microsoft to keep polishing "MS-DOS" until it becomes something like Windows 8 without starting again (twice).

You are starting from the assumption that their designs were good when they actually weren't. They may have seemed good at the time; that's another story.

They were (relatively) good designs at the time they were designed. For example, in the time before MS-DOS was released (before 80286 existed), it would've been unreasonable to expect Microsoft to develop a full multi-tasking OS with GUI, etc. For their next rewrite, Windows 9x was (given the need to support old DOS software that expected full control and no protection) also relatively good.

Should Microsoft have looked deep into their crystal ball, and designed DOS for GUI and 1920*1600 wide screens and systems with 8 or more 64-bit CPUs running at several GHz back when people were pushing just to get their 8 MHz CPU to handle simple (text mode) word-processing?

Also, notice that my main problem with that post was that the poster was advocating rewrites as a good practice.

Rewrites are good practice, when they're justified. Wasting years polishing a turd just because you're not smart enough to recognise the magnitude of design changes and/or because you lack the courage necessary to do what needs to be done, and then ending up with a unmaintainable pile of scar tissue is not good practice.

Of course rewrites are not good practice when they're not justified too; which is what makes it far too easy for people to take what I've been saying out of context.


Cheers,

Brendan

[Brendan]

Hi,

And while the 386 processor didn't exist initially when MS-DOS was written, it was on the market 1988 when I started my project. And I bet the 386 processor was at least at the early design-stages when MS-DOS was written.

The first version of MS-DOS was released in 1981. If it took 1 year to implement then it would've been designed in 1980. Back then (before Microsoft became historically important) they would've been lucky to hear vague rumours about the 80286 (which wouldn't have been released until 1982).


Cheers,

Brendan

[Love4Boobies]

@Brendan: I didn't read the whole post because it became even more obvious that there is a misunderstanding here. Even good designers sometimes fail---I accept that reality. You seem to think that by "a design that gets in one's way is a bad design" I really meant "people should be expected to always have good designs." What I'm actually saying is that this is what they should aim for and, should they fail, they should at least try to learn something from the experience. Again, I am contradicting the point that rewriting something because of a design change is good practice; I say that it is actually a sign that something went bad.

[linguofreak]

@rdos: Saying that MS-DOS was the biggest piece of junk is going overboard. Back then, computers were quite slow, expensive, had rudimentary graphics hardware, and there was no protected mode. There was also limited use for them, Microsoft didn't have many resources, etc. An OS like the one you descrined could not have existed for the PC back then.

But there were a multitasking system back then called CP/M (look here: http://en.wikipedia.org/wiki/CP/M). CP/M was a decent operating system, but for some reason IBM let Bill Gates do his hack and used that instead.

They actually approached Digital Research first, and DR asked for too much money.

If IBM had used CP/M instead, the PC market would be very different from how it is now, and it wouldn't have taken two major rewrites by Microsoft until there was decent multitasking on PCs.

The original version of CP/M was single-user. I also doubt that the multiuser variants of it *were* in fact decent multitasking systems (at least, not without hardware the IBM PC didn't have), because the 8086 had no means of enforcing protection: Any process could have taken over the system.

[bluemoon]

Back in the day, protection were not the #1 priority in multi-tasking system, but let the user perform multiple tasks smoothly (hot key pop out for TSR, or even in fg/ctrl-Z manner).

If there is a malware that take over the system, the user simply reboot and remove that software.

[Antti]

I just made a short video. This is my previous "OS" project (before "rewrite from scratch"):

http://www.youtube.com/watch?v=Hmxo10_ZN_s

It is easier to see the screen if you use the 720p or 1080p resolution. As you can hear, English is not my native language (I try to improve my pronunciation). This video is not very interesting because the OS itself is so basic. However, it would be really nice to see similar videos from users having something better to show.