OSDev in decline

[glauxosdever]

Hi,


I haven't thought about this extensively, but here are some random thoughts:

• When I joined in 2015, there was relatively a lot of activity both here on the forums and on IRC. I saw IRC members rising slightly until about 2017 or 2018. The forums, on the other hand, have visibly declined (even further after Brendan got banned) and now I see IRC too being mostly limited to a few regulars talking about borderline stuff.
• Some of my favourite OS projects have either lost interest from the audience or, worse, mostly abandoned. For example, ToaruOS doesn't spark now the same interest as it did three or four years ago. Sortix hasn't been committed to since at least two months.
• It's been suggested before that UEFI made the barrier to entry higher because of the lack of resources. I suspect this is true since, when I was trying to return to OSDev two months ago by doing a UEFI bootloader, I gave up due to the lack of time that was needed to look up everything needed.
• It's also been suggested that all the cool kids these days do webdev. But wouldn't this be roughly the same as 4 years ago when I joined? I don't think this explains the OSDev decline adequately.
• There are only three "relevant" desktop OSes these days, which makes it look like "normal" people can't write one. I remember some years ago looking at a market share graph that also included some BSD variants. Nowadays, even Linux has declined.
• The economic crisis possibly made people work more and thus not have time for time consuming hobbies like OSDev.

I'll probably try to write a good OS at this point to make more people interested in OSDev but, is there anything else that can be done to restore OSDev to its previous state?


Regards,
glauxosdever

[nullplan]

• It's been suggested before that UEFI made the barrier to entry higher because of the lack of resources. I suspect this is true since, when I was trying to return to OSDev two months ago by doing a UEFI bootloader, I gave up due to the lack of time that was needed to look up everything needed.

It's weird. With BIOS, we only had RBIL to look up things. Every BIOS worked a bit differently, and nothing was ever enforced, you just kind of had to hope that "int 13, function 2" means the same thing to the BIOS as it means to you. Now you have a public spec, and an entire SDK, and now you complain about a lack of resources? I don't get it.

• There are only three "relevant" desktop OSes these days, which makes it look like "normal" people can't write one. I remember some years ago looking at a market share graph that also included some BSD variants. Nowadays, even Linux has declined.

Yeah. The desktop PC has gotten less relevant, now that all the normies can just use phones and tablets for all their Internet surfing needs. It is still relevant for business, but the market still has shrunk.

Actually, developing for these devices might be fun (and I mean, developing my OS for these devices), but it is a walled garden, breaking out of it is hard (and will probably be patched), and more importantly: I rely on my phone a hell of a lot more than on my PC. If I break my PC, and can do without it pretty much indefinitely. If I break my phone, I'll need a replacement, STAT.

I'll probably try to write a good OS at this point to make more people interested in OSDev but, is there anything else that can be done to restore OSDev to its previous state?

Besides making people develop OSes? The optimist in me says this is a waveform, interest waned in the past, but it will come back at some point. Possibly. But the cynic in me says maybe this hobby is just destined to fade away. Either way, I won't try to predict the future.

[glauxosdever]

Hi,

• It's been suggested before that UEFI made the barrier to entry higher because of the lack of resources. I suspect this is true since, when I was trying to return to OSDev two months ago by doing a UEFI bootloader, I gave up due to the lack of time that was needed to look up everything needed.

It's weird. With BIOS, we only had RBIL to look up things. Every BIOS worked a bit differently, and nothing was ever enforced, you just kind of had to hope that "int 13, function 2" means the same thing to the BIOS as it means to you. Now you have a public spec, and an entire SDK, and now you complain about a lack of resources? I don't get it.

Indeed, having a public spec that forces implementations to behave the same way makes things more predictable and easier. On the other side, there is a lot of example code for various BIOS and other legacy stuff (not saying it's always of good quality though). Reading from the disk using INT 0x13, enabling the A20 line, switching to protected mode and others have been nicely documented on the wiki and we have a lot of example code both on the wiki and elsewhere. For UEFI stuff, the only example code I have found on the wiki are the Bare Bones tutorials and Uefi.inc (which is questionable). Around the Internet, there sure is some more stuff, but probably a lot less than for BIOS. But I'd guess it's just a reflection of the falling interest in OS development. Less people are interested in OS development these days, thus less people write about UEFI now than did about BIOS earlier.

Of course, however, I'm not saying there should be example code for everything, but it can often be useful when dealing with something complex.

• There are only three "relevant" desktop OSes these days, which makes it look like "normal" people can't write one. I remember some years ago looking at a market share graph that also included some BSD variants. Nowadays, even Linux has declined.

Yeah. The desktop PC has gotten less relevant, now that all the normies can just use phones and tablets for all their Internet surfing needs. It is still relevant for business, but the market still has shrunk.

That's true, however, I don't see many notable OSes for phones and tablets either.

I'll probably try to write a good OS at this point to make more people interested in OSDev but, is there anything else that can be done to restore OSDev to its previous state?

Besides making people develop OSes? The optimist in me says this is a waveform, interest waned in the past, but it will come back at some point. Possibly. But the cynic in me says maybe this hobby is just destined to fade away. Either way, I won't try to predict the future.

Of course, making people develop OSes is the point. But in order for people to be interested in this hobby, I think the following are needed:

• Some OSes that are interesting in some respect and have some online coverage. I was reading earlier today an article from 2008, about several alternative OSes that one could try. I feel like most of them have been forgotten by now. When I joined this community, Sortix and ToaruOS were often spoken about. I also spent a lot of time searching older posts. I remember seeing posts about Clicker, DexOS, MikeOS and others. Now I think, among these, ToaruOS is the only active OS and, to be honest, I don't see anything taking the place of the other ones.
• A slightly lowered entry barrier. I think that people some years ago were more patient than now, myself included. I can't imagine myself in the present time spending three months on a bad-quality EHCI driver, like I did in 2015, although that may be a reflection of me not having now as much time as I did back then.
• Anything else?

To fix the problem described in the first point, I started finding some free time to do some stuff, even if it's just the OS prerequisites (for now, a Version Control System). Hopefully, my OS could be interesting enough. As for the second point, when I get to the point of doing the actual OS, I'll try to contribute as much as possible to the wiki.

I hope this explains my stance on the matter.


Regards,
glauxosdever

[Kazinsal]

The biggest problem I see here is that you seem to be under the impression that ten years ago there were people who were writing the next desktop OS, when in reality most people were just writing extremely limited little toys that would spontaneously crash after a few hours or were artificially limited to 32 KB of RAM or something else.

The hobby was already miniscule to begin with, and the era of the hacker has been at death's door if not completely over for some time now. There were already few people who could actually get into the hobby and stick with it long enough to do something other than copy and paste a few broken tutorials together, and in recent years that number shrank significantly. It's not something we can really do anything about; 20 years ago there were more people experimenting with Slackware and the BSDs and the like that would find themselves in a position to get into OS development, possibly even contributing to one of those systems.

Brendan got himself banned for trying to be god-emperor of the OSDev.org forums and the site's owner wasn't interested in that. ToaruOS stopped aiming for "flashy port the everything" because that just gets you a GNU operating system on top of a custom kernel, and honestly, the rewritten goals of the OS are more interesting now than they were a few years ago. Sortix isn't worked on so much presumably because the author has real world stuff to do.

Writing a general purpose OS because you want to write something that rivals even one of the less commonly used general purpose OSes out there is a terrible idea that everyone wants to do and maybe only a dozen people in the hobby in the past 30 years have succeeded at, and I'd say half of those or more were 25+ years ago. People need to think of a problem they want to solve, and solve it. If that requires writing a bunch of bare metal code to do it, then hey presto, that's an operating system. If it doesn't...

I don't know what you expected. This hobby is statistically insignificant among programmers on the internet and these forums are just not a good representation of what the hobby is several years beyond the "copy and paste tutorials" stage.

[mikegonta]

It's the same old story.
There are those that write an operating system (UEFI notwithstanding) and then there's me - who write an operating system forum post.

[glauxosdever]

Hi,


I'm not saying people in here were writing the next desktop OS. But, given how there was a bigger diversity of OSes used by someone else other than the author (the BSDs, Amiga, BeOS, etc), it was probably more interesting to try to write an OS in the past than now.

But I definitely can see your reasoning. I noticed a decline since 2015, but it actually is part of a much larger decline since the 1990s. I think even researchers have said OS research has declined since then.


Regards,
glauxosdever

[coderTrevor]

For me, the barrier for alternative operating systems has always been a lack of drivers.

No matter how terrific a homebrew OS can be and how many standards it supports or how many software ports it can run, nobody can use the thing if it can't run on their hardware. Sure, you can run an OS in a virtual machine but it's just not the same IMHO.

I probably would have started writing an OS many years ago if I didn't know it was a completely futile effort. It's been on my bucketlist for a long time. Recently I decided I didn't care (because nobody will use my OS anyway) and just started writing my OS as a hobby, not as something I think will ever be used.

It makes me really sad and angry that UDI never caught on. Maybe because of Stallman's opposition, or maybe just because nobody bothered with it. I'm fairly certain it could have caught on if Linux embraced it but alas, that never happened. From my point of view Linux has just become a third proprietary OS requiring its own special set of drivers.

Of course all of this is just my personal take on the hobby and I'm very much a newcomer to OSDev. My views may have nothing to do with why anyone else avoids OS development these days, or why it was more popular in the recent past.

It may also be worth mentioning that in my personal endeavor with OS writing I haven't had much to say. I've been able to find answers to all of my questions here already, and I consider myself too new to really answer anyone else's questions. Maybe some of the "decline" is just that people don't want to rehash what's already been posted here?

P.s. I can't speak to any decline but I've seen some really cool projects here!

[iansjack]

I probably would have started writing an OS many years ago if I didn't know it was a completely futile effort.

If that's the way that you feel then it's probably a good decision.

To me OS development is just a hobby - a way of learning more about computers and programming. Plus a great sense of achievement when you accomplish what seemed like an impossible task. There's nothing futile about having fun.

But anyone who starts with the intention of writing a killer OS that is going to take over the world is crazy. It may happen - and you may win the lottery. But don't bank on it.

[Solar]

From my point of view Linux has just become a third proprietary OS requiring its own special set of drivers.

That is a quite correct observation. The fact that Linux drivers are "Open Source" benefits absolutely nobody except the Linux community. This was a quite deliberate decision of the kernel maintainers, and it (combined with the GPL) was about as lethal to operating system competition as Microsoft's software patents.

My views may have nothing to do with why anyone else avoids OS development these days, or why it was more popular in the recent past.

Apart from my own project biting the dust for organizational reasons (looooong ago), I found there are several things that would keep me away from "trying again" if I were in the same position as back then:

• Systems have become even more complex. Especially that virtually everything today is multi-core. This jumps at you very early in the boot process, and makes people realize very early on they might be in over their heads.
• Desktop isn't "the thing" anymore. Everybody is very much aware that the measliest mobile app gives you more bragging rights than the most polished hobby desktop OS -- and the barrier to toying around with a mobile the way we did with PCs "back in the day" is much higher. Also, hoping to "make things better" by delivering the greatest desktop OS there ever was has dropped down many places on the nerd's list of dreams.
• The "golden generation" of computing -- the guys for whom even the latest 9-iron of desktop machines is still merely a souped-up C64 in their heads, to be explored, mastered, and harnessed -- is slowly fading out. Most people today have never used anything but a x86, Windows, and perhaps Linux. This makes for a rather different mindset. IMHO, it made for a much larger percentage of people accepting the computer as an appliance to be used as-is, instead of something to toy around with. (Also see previous point.)

Combined with the reality of the driver situation -- and there only being ever more, and ever more complicated, devices to support -- it makes for a very bleak prospekt. OS development? What for?

Maybe some of the "decline" is just that people don't want to rehash what's already been posted here?

I want to believe that it's simply that the Wiki has become such a good resource over time that most people simply take what's in there, get started, and then ask their questions on StackOverflow instead of hanging around in this forum.

[Korona]

From my point of view Linux has just become a third proprietary OS requiring its own special set of drivers.

That is a quite correct observation. The fact that Linux drivers are "Open Source" benefits absolutely nobody except the Linux community. This was a quite deliberate decision of the kernel maintainers, and it (combined with the GPL) was about as lethal to operating system competition as Microsoft's software patents.

I am not so sure about that (i.e., that open driver only benefit Linux) - for example, writing my driver for Intel graphics mode setting (and the corresponding wiki page) would not have been possible without looking at the Linux sources. The official documentation is just not precise and complete enough to write a driver. The same applies where the specifications are behind paywalls (the MCFG ACPI table comes to mind).

If anything, Linux might have resulted in less desire to write a new OS because it is considered to be "good enough".

[Solar]

If anything, Linux might have resulted in less desire to write a new OS because it is considered to be "good enough".

{hysterical laughter}...

[pat]

I'll probably try to write a good OS at this point to make more people interested in OSDev but, is there anything else that can be done to restore OSDev to its previous state?

I had an idea while heading home from work today. If you're looking to spark up interest, howabout introducing a weekly/biweekly/monthly thread where people talk about what they've recently accomplished, or their short (current month at most) term and concrete plans? Sort of like a book group except not.

The screenshot thread is great and always inspiring to browse, but discussion is limited. This would be a more indepth thread where you'd be able to go into more detail about one key thing you've added which might stimulate conversation on the pros and cons, alternate ideas and whatnot. Why'd you do this, have you thought about that and so forth.

Personally I'd find that to encourage myself to put in a bit more effort, both to have something for "show and tell", but also friendly rivalry - oh $ExampleOS recently added a new $driver, I bet I can add that to $MyOS in short order. We'd be pushing eachother to continue our projects, while generating ideas that might get others interested.

[bzt]

Hi

I think Korona is right that Open Source serves more people. I for one couldn't have used the RPi's undocumented hardware random generator if it weren't for the kernel source.

On the other hand, Solar is totally right about the quality of the Linux kernel, it's a huge mess. My finger get's numb just by scrolling through this page, it's so long: https://github.com/google/syzkaller/blo ... nd_bugs.md.

Finally to the OP: I don't think OSDev is declining. I think the whole industry is declining in an alarming rate. I used to teach at a university and I remember very well when the first students from this new era arrived. Before that there were no more than 6-8 people in a group, all of them were motivated, interested in programming and willing to learn on their own in their free-time. After that the number of students per group tripled, because everybody though (and still falsely thinks) that anybody can be a programmer. But they are not motivated at all, they are not willing to learn on their own. They're expecting the univerisity to hand over the knowledge required to get a well-paid job, but that's all. I had a feeling if they were told that a barber had a better wage they would have gone there. They expect to have a point'n'click game like interface for programming, without the slightest understanding what's going on under the hood. Again, this is not only OSDev related. For example once I had to work with Java web "developers" who had absolutely no clue what HTTP is and how it works. They just knew that their huge 1G (no mistake, 1G) middleware code will call a method in a class for an url. This was fine until a bug messed up one of the get query stings, and the whole department was just sitting clueless without the slightest idea where to look. After a week I felt sorry for them and I run a tcpdump, construct a tcp-stream from the packets to show them what went wrong (this was many years ago, before browsers had a built-in inspector). They looked at me if I was some kind of sorcerer or something...
With the "A" and "i" tablets and mobile markets this got even worse, because now many programmer-wannabes click together "applications" to sell, and they quite often include malicious code in their fine "products" just because they don't know what library they're using.

No wonder that there are relatively less OSdevers. Or more precisely the real talents are now lost in the increased noise.

Cheers,
bzt

[glauxosdever]

Hi,

I had an idea while heading home from work today. If you're looking to spark up interest, howabout introducing a weekly/biweekly/monthly thread where people talk about what they've recently accomplished, or their short (current month at most) term and concrete plans? Sort of like a book group except not.

I think this is a very good idea. I would however recommend it isn't bound to some time range (just like you can post in the Screenshots thread any time). I would also suppose something like this would attract plans that include some questions. (Should that be allowed? I'd say yes.)

Now, who is willing to begin this thread? Probably not me, as I don't have an established OS currently, just some plans for its building/hosting prerequisites. Still debating with myself whether I can find some time currently and whether I should use files or a database for the VCS that will be used to host the OS.

This was fine until a bug messed up one of the get query stings, and the whole department was just sitting clueless without the slightest idea where to look. After a week I felt sorry for them and I run a tcpdump, construct a tcp-stream from the packets to show them what went wrong (this was many years ago, before browsers had a built-in inspector). They looked at me if I was some kind of sorcerer or something...

What a coincidence you said that just today, as we had an issue in the database labs in the morning. We weren't able to connect to the MySQL server that was running locally on each computer. This went unsolved for the duration of the lab, as no one could fix it, including the professor. The result was that we couldn't write SQL queries ourselves, just watch the professor type them (it worked on her laptop). Too good for me that I had downloaded and launched SQLite instead and created the database tables manually, instead of extracting an Oracle Data Modeler design to some Oracle SQL and then converting this Oracle SQL to MySQL syntax, then run it on the server. I could at least type some of the queries shown after finishing creating the tables. I don't want to comment further on that.


Regards,
glauxosdever

[Schol-R-LEA]

If anything, Linux might have resulted in less desire to write a new OS because it is considered to be "good enough".

{hysterical laughter}...

Well, if the bar is set low enough... and certainly the guys in Redmond have done their fair share to lower expectations themselves.