Is asynchronous IPC preventing us from using parallelism?

Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by rdos »

Combuster wrote:Most potential users are already using a message queue. It's the accepted standard for doing GUIs.
Yes, but not for accessing files.
OSwhatever
Member
Member
Posts: 595
Joined: Mon Jul 05, 2010 4:15 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by OSwhatever »

Combuster wrote:Most potential users are already using a message queue. It's the accepted standard for doing GUIs.
GUIs usually don't need a response, just sending an event. Asynchronous messages are therefore very well suited for that purpose.
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by rdos »

berkus wrote:
rdos wrote:
Combuster wrote:Most potential users are already using a message queue. It's the accepted standard for doing GUIs.
Yes, but not for accessing files.
You've missed the point of opening not 1 file but thousands (which is just an example of the concept by the way).
I accept that Brendan's solution is better if you want to open 1000 files, but typical applications use LIBC to open one file at a time, and therefore has no reason to use an aynchronous interface in the first place, and if wrappers that have the same performance as non-asynchronous APIs cannot be constructed, most users will go with the standard solution instead. Not only that, but there is no standardized API for doing asynchronous file operations, so each implementation would likely be incompatible with every other implementation, again leading to few people using the functionality, and if they require it, use some common implementation like the Win32.
User avatar
Griwes
Member
Member
Posts: 374
Joined: Sat Jul 30, 2011 10:07 am
Libera.chat IRC: Griwes
Location: Wrocław/Racibórz, Poland
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Griwes »

rdos wrote:I accept that Brendan's solution is better if you want to open 1000 files, but typical applications use LIBC to open one file at a time, and therefore has no reason to use an aynchronous interface in the first place, and if wrappers that have the same performance as non-asynchronous APIs cannot be constructed, most users will go with the standard solution instead. Not only that, but there is no standardized API for doing asynchronous file operations, so each implementation would likely be incompatible with every other implementation, again leading to few people using the functionality, and if they require it, use some common implementation like the Win32.
You seem to be saying that there is no reason to create good (and possibly portable*) asynchronous API, because there is no standardized asynchronous API out there... o.O


*I mean, implemented also on other platforms than its native one, using those platform's native APIs.
Reaver Project :: Repository :: Ohloh project page
<klange> This is a horror story about what happens when you need a hammer and all you have is the skulls of the damned.
<drake1> as long as the lock is read and modified by atomic operations
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Brendan »

Hi,
rdos wrote:I accept that Brendan's solution is better if you want to open 1000 files, but typical applications use LIBC to open one file at a time, and therefore has no reason to use an aynchronous interface in the first place, and if wrappers that have the same performance as non-asynchronous APIs cannot be constructed, most users will go with the standard solution instead. Not only that, but there is no standardized API for doing asynchronous file operations, so each implementation would likely be incompatible with every other implementation, again leading to few people using the functionality, and if they require it, use some common implementation like the Win32.
I'm not sure about you, but I'm writing my own OS (and therefore I get to choose my own standards and develop my own APIs). I don't care about standards or APIs intended for other OSs. In the same way, I couldn't care less what LIBC does (or that LIBC is unable to take advantage of my OS's features) because I have no intention of supporting or using LIBC anyway.

Basically, I am not reinventing a wheel, I'm creating something "new" (like caterpillar tracks) as a completely different alternative to wheels. You can complain that standard hubcaps won't fit, but while you're complaining I'll be doing this:

Image

8)

Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by rdos »

Griwes wrote:You seem to be saying that there is no reason to create good (and possibly portable*) asynchronous API, because there is no standardized asynchronous API out there... o.O
I mean it seems to be to do things in the wrong order. First, define and document an asynchronous file-API, and get it accepted as a standard. Then implement it in your OS. By first doing something native in your OS you are almost garanteed to not get it accepted as a standard that others will try to implement. Unless you are Microsoft, in which case you can rely on people using it by virtue of many users.
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by rdos »

Brendan wrote:I'm not sure about you, but I'm writing my own OS (and therefore I get to choose my own standards and develop my own APIs). I don't care about standards or APIs intended for other OSs. In the same way, I couldn't care less what LIBC does (or that LIBC is unable to take advantage of my OS's features) because I have no intention of supporting or using LIBC anyway.
FYI, I first documented my network IPC protocol (Simple Messaging Protocol, SMP) and registered it, and then I implemented it in my own OS. While WireShark doesn't show the exact format of my registered IP-protocol, it does tag it as "SMP". I could also have written RFCs about the protocol, and possibly gotten those registered as well, and then it would have been fully open and reusable. I think that is the preferred way you go about new APIs.
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Brendan »

Hi,
rdos wrote:FYI, I first documented my network IPC protocol (Simple Messaging Protocol, SMP) and registered it, and then I implemented it in my own OS. While WireShark doesn't show the exact format of my registered IP-protocol, it does tag it as "SMP". I could also have written RFCs about the protocol, and possibly gotten those registered as well, and then it would have been fully open and reusable. I think that is the preferred way you go about new APIs.
Are you sure you understand the difference between a networking protocol and an Application Programming Interface (API)?

I just put the muesli in a bowl, pour milk on top of it, then sprinkle a little sugar on top. That's the preferred way to go about making a pizza. #-o


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
User avatar
Griwes
Member
Member
Posts: 374
Joined: Sat Jul 30, 2011 10:07 am
Libera.chat IRC: Griwes
Location: Wrocław/Racibórz, Poland
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Griwes »

rdos wrote:
Griwes wrote:You seem to be saying that there is no reason to create good (and possibly portable*) asynchronous API, because there is no standardized asynchronous API out there... o.O
I mean it seems to be to do things in the wrong order. First, define and document an asynchronous file-API, and get it accepted as a standard. Then implement it in your OS. By first doing something native in your OS you are almost garanteed to not get it accepted as a standard that others will try to implement. Unless you are Microsoft, in which case you can rely on people using it by virtue of many users.
If your API is good, easy to use and makes sense in every aspect, someone (be it another hobbyist or anyone else) will eventually implement it on top of every possible native API, making it portable; and, if you designed and implemented it well, it will still have decent performance, even if it's just a wrapper over native API, in cases of OSes other than the API's native one.

And, I don't care what MS does; I assume you know that not everything running and working on Windows is written by Microsoft? Plus, what Brendan said, it's your OS, you shouldn't really care about others. You should design your API and implement it to get it right, not to be first accepted - if it is good, it will be accepted even if you first implement it (even as a proof of concept) and then document in form of RFC; if it is lame, no-one will ever implement or use it, even if you first write RFC.

Also, when you design an API that becomes fully portable, no-one will have any interests of using your OS - and isn't providing unique useful APIs one of ways to get some attention for your OS?
Reaver Project :: Repository :: Ohloh project page
<klange> This is a horror story about what happens when you need a hammer and all you have is the skulls of the damned.
<drake1> as long as the lock is read and modified by atomic operations
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by rdos »

Griwes wrote:And, I don't care what MS does; I assume you know that not everything running and working on Windows is written by Microsoft? Plus, what Brendan said, it's your OS, you shouldn't really care about others. You should design your API and implement it to get it right, not to be first accepted - if it is good, it will be accepted even if you first implement it (even as a proof of concept) and then document in form of RFC; if it is lame, no-one will ever implement or use it, even if you first write RFC.
That is true. In order to become used, it must both be good, and documented. :mrgreen:
Griwes wrote:Also, when you design an API that becomes fully portable, no-one will have any interests of using your OS - and isn't providing unique useful APIs one of ways to get some attention for your OS?
I think that is wrong. In order for your OS to be usable with free software, it must come with a portable layer (for instance LIBC). The existence of LIBC support does not mean the native API couldn't contain some extra powerful features, or some extra protocols that provide better functionality.
Last edited by rdos on Mon Jun 04, 2012 7:51 am, edited 2 times in total.
Ready4Dis
Member
Member
Posts: 571
Joined: Sat Nov 18, 2006 9:11 am

Re: Is asynchronous IPC preventing us from using parallelism

Post by Ready4Dis »

rdos wrote:
Griwes wrote:And, I don't care what MS does; I assume you know that not everything running and working on Windows is written by Microsoft? Plus, what Brendan said, it's your OS, you shouldn't really care about others. You should design your API and implement it to get it right, not to be first accepted - if it is good, it will be accepted even if you first implement it (even as a proof of concept) and then document in form of RFC; if it is lame, no-one will ever implement or use it, even if you first write RFC.
[/qoute]

That is true. In order to become used, it must both be good, and documented. :mrgreen:
Griwes wrote:Also, when you design an API that becomes fully portable, no-one will have any interests of using your OS - and isn't providing unique useful APIs one of ways to get some attention for your OS?
I think that is wrong. In order for your OS to be usable with free software, it must come with a portable layer (for instance LIBC). The existence of LIBC support does not mean the native API couldn't contain some extra powerful features, or some extra protocols that provide better functionality.
The problem is, if you make the compatibility layer, it's just like every other OS, so what's the point? If someone is writing software for free OS's, and all the OS's they want to support use standard libraries, they will never use your specific features that make your OS better, meaning it's just another *nix clone. If people move to your OS because it's not a *nix clone and it does something better, then they will not just use the compatibility layer (since one doens't exist), but use your OS for what it was actually intended.
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Combuster »

rdos wrote:In order for your OS to be usable with free software, it must come with a portable layer
I have UDI. Therefore I have proof that portable free software can run on a pure asynchronous IPC API.

Free software doesn't equal GNUs - the latter prefer relatively barren lands :mrgreen:
Last edited by Combuster on Mon Jun 04, 2012 8:17 am, edited 2 times in total.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
rdos
Member
Member
Posts: 3297
Joined: Wed Oct 01, 2008 1:55 pm

Re: Is asynchronous IPC preventing us from using parallelism

Post by rdos »

Ready4Dis wrote:The problem is, if you make the compatibility layer, it's just like every other OS, so what's the point?
You don't need to support the compability layer natively. You can just first define your own API (keeping in mind that you want to be able to support LIBC at some point), and then you implement LIBC with the native API. The native API can be widely different from LIBC, as long as all the relevant parts of LIBC can be supported to the desired extent.
User avatar
Griwes
Member
Member
Posts: 374
Joined: Sat Jul 30, 2011 10:07 am
Libera.chat IRC: Griwes
Location: Wrocław/Racibórz, Poland
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Griwes »

rdos wrote:
Ready4Dis wrote:The problem is, if you make the compatibility layer, it's just like every other OS, so what's the point?
You don't need to support the compability layer natively. You can just first define your own API (keeping in mind that you want to be able to support LIBC at some point), and then you implement LIBC with the native API. The native API can be widely different from LIBC, as long as all the relevant parts of LIBC can be supported to the desired extent.
Again, *you* don't need the compatibility layer. If anyone wants to use your OS, he will do it only if you have something new to show him; if he decides to use standard C library as well, he can just write those functions required to port the library to given OS and use it as compatibility layer - but there is no need for *you* to do it.

People won't use new API, if there is old API also available, because they tend to prefer coding using known things rather than learning new APIs. And if they will not use your brand new API, they will not use your OS, because the old API is also everywhere and your OS doesn't give them anything unique.
Reaver Project :: Repository :: Ohloh project page
<klange> This is a horror story about what happens when you need a hammer and all you have is the skulls of the damned.
<drake1> as long as the lock is read and modified by atomic operations
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: Is asynchronous IPC preventing us from using parallelism

Post by Combuster »

And somehow again we went from a fair topic question to teaching rdos facts and reasoning.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
Post Reply