Sockets as a filesystem?

[Primis]

The unix philosophy of "treat everything as a file" doesnt truely apply to sockets, however I'm under the impression that plan 9 has managed to implement a socketFS.
Is there any disadvantages to doing so? Also, how would I begin to do somthing like it? I'm not asking for a tutorial here, more like a working explanation I can wrap my head around.

I understand the concept of special files and the like, but web sockets seem a bit too... Well, far away from a file.

[iansjack]

The unix philosophy of "treat everything as a file" doesn't truly apply to socket .... I understand the concept of special files and the like, but web sockets seem a bit too... Well, far away from a file.

Could you perhaps expand on those statements?

[SpyderTL]

The reason that you would "treat" everything like "one" thing is that something will need to "use" that thing in the future, and trying to pre-define every different type of thing would (literally) take forever. So, the easiest approach is to pre-define one thing, and then make everything act according to those pre-defined rules.

That "thing" that you define can be anything you want: a file, a pipe, a stream, an object, a widget. You just have to define what that thing is/can do, and then you have to figure out how to "force" all of the real world things into that one "thing".

If you do it right, then you only have to write your code once, and it will work for any situation.

Also, keep in mind that naming your stuff is simply for your benefit, and perhaps the benefit of your users. The machine could care less what you call it.

EDIT: Also, keep in mind that the Linux "file" paradigm is only used on the user facing end of the OS. Behind the "file" facade, you also have "modules", "drivers", "consoles", "devices", "processes", etc. So even Linux doesn't really stick to one definition of a "thing".

[bluemoon]

The main operations on file include: open, close, read, write and seek (thanks alexfru). This pretty much covers everything in the world, or do they?

Then you have special cdrom that acts as file, but you need to support a function to eject the disc.
Then you have linear frame buffer that acts as a file, but you need a way to adjust video mode.
Then you have special sound card that acts a a file, but you need to provide device control function like volume and mechanism for injecting audio filters.

Then you want to have event-based notifications beside the basic read/write (e.g. use press the eject button).

All those extra control is not unified into the file concept (although you assign function in ioctl, it just doesn't make sense to eject disc on a sound card).


On Linux, the OS give up and the file concept works just like IPC. You then open up a data transfer file and a control file and do it yourself for everything other that a real file.

[alexfru]

The main operations on file include: open, close, read, write. This pretty much covers everything in the world, or do they?

lseek?

[iansjack]

The main operations on file include: open, close, read, write. This pretty much covers everything in the world, or do they?

lseek?

I'd disagree with that one.

I guess it all depends upon what you call a "file" but to my mind you can only perform seek operations on a random-access file. (The same would be true for pipes.) The concept of a sequential file, which can only be read from start to end (although perhaps you can allow for forward-only seeks that ignore parts of the file), is a useful one. I agree with bluemoon's other operations, and sockets support all of them; hence my request for a clarification from the OP as to in which way the file designation is not appropriate for sockets.

[alexfru]

The main operations on file include: open, close, read, write. This pretty much covers everything in the world, or do they?

lseek?

I'd disagree with that one.

I guess it all depends upon what you call a "file" but to my mind you can only perform seek operations on a random-access file. (The same would be true for pipes.) The concept of a sequential file, which can only be read from start to end (although perhaps you can allow for forward-only seeks that ignore parts of the file), is a useful one.

I didn't say it wasn't useful. But there are a lot of important file formats and software that need random access to files. You may argue that one could read the whole thing into RAM and do in-memory seeks, but that's not something always practical. So, I wouldn't discard lseek.

[iansjack]

I'm not suggesting that you discard lseek as an operation on (random-access) files, just that it is not a necessary operation on any file.

The reason that I think sockets are validly considered as files is because that allows them to be redirected to STDIN and STDOUT.

[alexfru]

I'm not suggesting that you discard lseek as an operation on (random-access) files, just that it is not a necessary operation on any file.

The reason that I think sockets are validly considered as files is because that allows them to be redirected to STDIN and STDOUT.

No objection there.

[bluemoon]

I'm not suggesting that you discard lseek as an operation on (random-access) files, just that it is not a necessary operation on any file.
The reason that I think sockets are validly considered as files is because that allows them to be redirected to STDIN and STDOUT.

However, as in CDROM and sound card cases, sockets is more than file. You have connections, events, windowing, dynamics, and other consideration. It would be better to say socket is not exactly to design as a file, but it can inherent the basic file operations and manipulated like a file in part of its life cycle.

[iansjack]

However, as in CDROM and sound card cases, sockets is more than file.

But surely it is true to say that any file is more than a (generic) file? For example, with a random-access file on a disk you can seek it, which you cannot necessarily do with a generic file. (I think this is where we came in.)

[Primis]

To expand a little,
Plan 9 has a /net directory, and rather than use ioctls to open and close new sockets, you write to a special file called /net/tcp or /net/udp to create sockets.
Plan 9 literally has everything as a file. And unfortunately, the plan 9 website is down (http://plan9.bell-labs.com) and there isn't much other information on their implementation.

[SpyderTL]

There have been several threads about different "file" systems, and how they try to "organize" things. I have a bit of a problem with "files" as a base object definition, especially when it comes to personal OS development.

As far as I can tell, the concept of "files" was an attempt to help users understand the concept of organizing data on a storage device by "emulating" a file cabinet, with file folders, and documents. But the concept doesn't hold up very well in the real world, where documents can be shared between people and applications, indexed in different ways.

As just one example, how many times have you run a music application that immediately wants to reorganize all of your music files into a specific folder structure? How about multiple users? What if two users want their music organized differently?

The file/folder paradigm doesn't hold up well to these types of requirements. If you are creating your own OS, why stick to "traditional" solutions to these problems. Try thinking outside the box before you decide to go down that path.

[Schol-R-LEA]

@SpyderTL: are you familiar with Project Xanadu and the concepts behind xanalogical document storage? One of my primary goals in Kether is to implement xanalogical storage (independent of the current PX code base, but with the idea licensed from Nelson) rather than a file system.

For those unfamiliar with it - which will probably be the majority of those reading this - the idea is simple, but difficult to implement, and often poorly explained (including by Ted Nelson, sadly; while he's good at inspiring enthusiasm, he has a lot of trouble getting the idea across). From a technical perspective, the core of a xanalogical system is a collection of out-of-band internal hyperlinks to fragments of data, each of which holds the information about the source, creation date, size, ownership, visibility and format of the data fragment. Each document consists of several such links.

Sounds sort of like an i-node, right? Well, not really. You see, the data fragments do not form a single document, but are independent of each other, and multiple links can point to a part of any given fragment. Documents in xanalogical storage are logical entities, not structural ones. There is no master directory of documents; rather, documents are built dynamically when requested, and any document can include and share something originally created in another document - a process called transclusion - seamlessly to form a new document.

Data segments are stored permanently in a single (logical) address, and never permanently copied, though they may be mirrored for backup purposes, and may be temporarily copied to a LRU cache for remote access. Copies, whether permanent or temporary, have the same logical address as the original. Networked copies have the same visibility and publication status as the original, and can be served from any existing copy if it is more readily available than the original (such that if a request for a data segment is sent through the network, each system it passes through should check to see if a copy is held locally, and should short-circuit the fetch to use that copy rather than going on to the originating system).

When a document is created, the data in it is journaled, and periodically saved as a new data segment. When a data segment that has been already stored is edited, rather than changing the existing data segment, a new data segment is created with an alternate link to the new segment. This means that (to an extent) there is a record of significant changes to the document, and (among other things) continuous undo for changes.

The hyperlinks are not directly visible, as Web hyperlinks are, but rather are a means by which documents are structured and presented. Presentation of the links themselves is a front-end function, but in most cases they are a back-end entity. The links are bi-directional ('bi-visible' in Xanadu terms), meaning that the presentation software can (if it has permission to and is written to do so) allow the browsing user to follow the link back to the originating document.

I'm a bit short on time right now, but that should give some flavor for it. I can explain more later if needed.

[embryo2]

From a technical perspective, the core of a xanalogical system is...

It looks like a graph of chunks of data. And above the graph there is a mechanism that joins some chunks according to some rules. Right?

If yes, then we have just the joining rules beside of the well known graph data structure. Generally speaking, we can select a subgraph and say - it's a new document or whatever we are interested in. But what's new we can see in such approach? Only the old fashioned graph and it's parts. Or is there something special?