Everything is an Object

[Marionumber1]

Operating systems are responsible for managing system resources. An important part of this how these resources are exposed to userspace. Unix-based operating systems follow a model where most resources, such as files and directories, pipes, sockets, and devices, are exposed as files. However, despite the flexibility this model brings, it also introduces problems, namely that the file interface doesn't work for every resource. An idea I like better comes from Windows, where all system resources are represented as objects, but I feel that they didn't take this concept as far as they could have. So I'd like to propose an idea similar to that where everything is an object.

My idea is based on three major concepts: objects, interfaces, and servers. All services and resources are exposed as objects, which implement interfaces. Interfaces are groups of methods that can be called. Each object can implement one or more interfaces, allowing it to expose different sets of services. Finally, all objects are hosted in servers. Servers are executable code modules that implement objects. They can be in-process servers (shared libraries), user-process servers (both local and remote), and kernel-mode servers (implemented by kernel-mode components).

Each interface must implement a set of generic object methods. Generic object methods are functions for querying other interfaces, reference counting, and access control. The function for querying interfaces provides access to other interfaces that a class defines. Reference counting functions are used to allow the object to be deleted when it is no longer in use. Access control allows an object's access permissions to be queried and set.

Userspace code is able to access objects through a component model (similar to COM and CORBA). The component model consists of a runtime library and language bindings. The runtime library implements features required to use objects. They provide code for recognizing servers, instantiating objects, and managing communications between the client (the code using the object) and the server. Servers are recognized through a global registration mechanism, where they specify which classes they support. Language bindings map the component model between languages. One idea of the component model is support by multiple languages. It mainly handles translation of data types between the client and server.

The way objects are used depends on which type of server they're hosted in. For in-process servers, the client simply calls an object's methods directly, since both reside in the same address space. For user-process servers, it gets more complicated. The client still calls code within its address space, but it's actually calling a proxy object generated by the component runtime. The proxy's methods really result in a message being sent to the server (this messaging is a service implemented by the kernel). For kernel-mode servers, the process is somewhat more complicated. A proxy object is used, but rather than sending a message to another process, the runtime calls a kernel syscall to make the method call.

What happens next? The syscall enters a kernel component called the object manager. The object manager is a kernel component that manages all of the system resources resources as objects. System resources include files, directories, processes, threads, IPC primitives, and devices. All resource access goes through the object manager. Each object managed by the object manager gets access to reference counting and security services. Objects are exposed to userspace through handles. They are created by the object manager whenever an object is opened. A process must own a handle to an object before using it. So basically, the proxy object wraps the handle generated by the object manager. The syscall into the object manager takes the handle, method number, and method parameters as arguments. The object manager converts the handle into an object, looks up the method within that object, calls it, and returns to userspace. Now the proxy object's method can return the results.

In addition to this, objects managed by the object manager can be given a name to identify it. The object manager maintains an internal object namespace that organizes named objects in a hierarchy. This allows for objects to be categorized and opened in a uniform matter. Namespaces like the filesystem namespace become part of the object namespace. The only way for userspace to gain access to objects is through the object namespace. User applications can open objects in the namespace and get a handle to that object. When userspace code opens an object, it requests a specific interface. The namespace system is integrated with the component model, allowing for all named kernel objects to be opened as component model objects.

I believe this creates a very powerful and flexible system. Objects can be transparently instantiated and accessed transparently, whether they reside in the client's address space, in another process (local or remote), or in the kernel. This makes it much easier for services (such as the window manager) to be implemented in remote processes. And rather than having resources exposed as files, they can expose their own sets of methods. For example, a graphics device could expose several different interfaces to userspace: a file interface (allowing it to be treated as a file), a framebuffer interface, a 2D acceleration interface, and 3D acceleration interface. What do you all think of this system? Feel free to provide feedback below.

[AbstractYouShudNow]

That would be interesting, since your drivers could be implemented in either user-space or kernel-space, or both. However, I think you would lose quite a lot of time in system calls and interprocess communication. That being said, if your kernel is a microkernel, you will benefit much from that model.

One last thing I would add is that you didn't mention security in your description. I think you should add some restrictions on the access to objects, since some objects should nnot be accessible by everyone. For example, a graphics driver's object should only be accessible by the GUI server, and the disk driver should only be accessible to the VFS server or programs that are granted the "raw disk access" priviledge (or simply administrator priviledges in Windows).

Have you started to implement it already ?

[Marionumber1]

That would be interesting, since your drivers could be implemented in either user-space or kernel-space, or both. However, I think you would lose quite a lot of time in system calls and interprocess communication. That being said, if your kernel is a microkernel, you will benefit much from that model.

Yeah, supporting userspace and kernel mode drivers was one of the reasons I came up with this idea. I don't think there'd be that much of a performance loss. Calls to objects hosted inside in-process servers can easily be done, and that will constitute a lot of calls. Calls to kernel-mode servers would require syscalls anyway, so that can't be avoided. You are right that calls to user-process servers could lead to overhead, but as long as my messaging system is implemented well, performance should be suitable. I don't actually have a microkernel, my kernel is more of a hybrid kernel, but I think this idea would still work well.

One last thing I would add is that you didn't mention security in your description. I think you should add some restrictions on the access to objects, since some objects should nnot be accessible by everyone. For example, a graphics driver's object should only be accessible by the GUI server, and the disk driver should only be accessible to the VFS server or programs that are granted the "raw disk access" priviledge (or simply administrator priviledges in Windows).

I actually did mention access control methods as part of the generic object methods. The idea was that objects could be secured by access control lists (ACLs). The object would provide routines to manipulate them. How those routines are implemented would be up to the object. Most kernel objects would simply use the routines provided by the kernel security manager. The idea behind how ACLs work is similar to what you said. ACLs contain entries for users, groups, and processes. These entries would explicitly allow or deny access to the users, groups, or processes that access them. Just like you said, I could limit object access to certain processes that are allowed.

Have you started to implement it already ?

Not yet, I want to work out the possible kinks before I start writing this.

[Jezze]

The first part sounds very much like what a normal device driver interface would look like and the rest sounds very similar to Plan9 and it's 9P protocol except it uses objects instead of files. It sounds pretty solid in theory but there might be a bunch of problems that show up because it requires a lot of implementation work before you can really show it works and therefor it is very hard at this stage to argue the benefits.

You could say that files are pretty good at representing resources and they can in fact represent every type of resource in a system but they are certainly not the most efficiant way to do it in most cases. What all resources have in common is that you want to able to read and write data from them which files can handle perfectly well. Then its up to the implementer of a resource to decide what granularity they want. Should a file represent a structure of data or a single value or something inbetween. This is where files have no clear set of rules while with objects you don't really need to think about it.

The reason why files are used and not objects is that because if you add new objects to the system, every program that doesn't know about this new type needs to be updated to be able to use them, you dont have that problem to the same extent with files. Have you thought about that?

Have you thought about how you will navigate resources? Will you query them like SQL or present them hierarchaly like a normal filesystem? Why I am worried here is that if you unify all resources into one huge database it could become very slow.

[Brendan]

Hi,

The reason why files are used and not objects is that because if you add new objects to the system, every program that doesn't know about this new type needs to be updated to be able to use them, you dont have that problem to the same extent with files. Have you thought about that?

That's not quite right.

In systems that use "files" for device driver interfaces you end up with a whole bunch of problems (caused by lacking functionality) because devices aren't files at all and need to support device specific things that are more than just transferring data. This is followed by IOCTLs as a work-around for the missing functionality problem, followed by "every program that doesn't know about this new set of IOCTLs needs to be updated to be able to use them".

Mostly; you can have standardised file IO functions or standardised data transfer methods and it's basically the same thing with different syntax (e.g. "read(handle)" vs. "handle::read()"); and you can have device specific IOCTLs or device specific methods that are still basically the same thing with different syntax (e.g. "ioctl(handle, CDROMEJECT)" vs. "handle::CDROMEJECT()"). You can even have wrappers to make one look like the other, and these wrappers would do virtually nothing because it's the same underlying functionality in both cases anyway.


Cheers,

Brendan

[Marionumber1]

The first part sounds very much like what a normal device driver interface would look like and the rest sounds very similar to Plan9 and it's 9P protocol except it uses objects instead of files. It sounds pretty solid in theory but there might be a bunch of problems that show up because it requires a lot of implementation work before you can really show it works and therefor it is very hard at this stage to argue the benefits.

Yeah, I wish that the merits of the idea would speak for themselves, but instead, we got COM and CORBA, which are barely used today except on legacy codebases.

You could say that files are pretty good at representing resources and they can in fact represent every type of resource in a system but they are certainly not the most efficiant way to do it in most cases. What all resources have in common is that you want to able to read and write data from them which files can handle perfectly well. Then its up to the implementer of a resource to decide what granularity they want. Should a file represent a structure of data or a single value or something inbetween. This is where files have no clear set of rules while with objects you don't really need to think about it.

Indeed, that's one of the reasons I'm using objects rather than files. The main problem with files is that there's no formally defined "contract" required to access them.

The reason why files are used and not objects is that because if you add new objects to the system, every program that doesn't know about this new type needs to be updated to be able to use them, you dont have that problem to the same extent with files. Have you thought about that?

As Brendan said above, files have a similar problem. Despite files being touted as something that's understood by all applications, that only applies to the standard file syscalls. ioctl() has the exact same problems as using an interface. Now in practice, this won't be a problem in my system. First of all, all objects can expose multiple interfaces, so a device can expose a file interface and be treated like a file. Second, most applications will be accessing a specific class of devices (and therefore will be accessing a constant interface). If a new object type is created, then someone writes a new program for it.

Have you thought about how you will navigate resources? Will you query them like SQL or present them hierarchaly like a normal filesystem? Why I am worried here is that if you unify all resources into one huge database it could become very slow.

In my original post, I mentioned a kernel component called the object manager. The object manager maintains an internal object namespace, similar to the filesystem, that organizes named objects in a hierarchy. Namespaces like the filesystem namespace actually become part of the object namespace. Since then, I've extended my idea somewhat. My current idea is that certain servers can act as name servers, which work together to build a unified object namespace. Each name server implements a part of this namespace, with the object manager being the first one to build it. Other servers could help build this namespace as well; for example, the window manager could create an object directory called \Windows to hold the window objects associated with each process.

[rdos]

It basically sounds like a reasonable way to do it, except that I think that the central object manager is a flawed concept which would lead to the same problems that files/IOCTL and Windows handles have. The thing is, you don't want one common object model, you want one object model per device-type. So you want to have a file interface with methods adapted for files, a directory interface with directory-related methods, a sound interface to playing sounds, a graphic interface, a mouse-interface, a touch-pad interface, a socket interface and so on. You should define standard object interfaces for these basic classes, and then you provide physical adaptations for those, and thus gain device-independence.

The way I do it in my OS, is that I have a module that implement handles (creating, dereferencing, deleting), but that doesn't know the purpose of the object it protects. Then I have a list of handle-types which are associated with each handle. When a handle is dereferenced, the handle type is passed and checked, which makes sure that handles are of the correct type.

[Marionumber1]

It basically sounds like a reasonable way to do it, except that I think that the central object manager is a flawed concept which would lead to the same problems that files/IOCTL and Windows handles have. The thing is, you don't want one common object model, you want one object model per device-type. So you want to have a file interface with methods adapted for files, a directory interface with directory-related methods, a sound interface to playing sounds, a graphic interface, a mouse-interface, a touch-pad interface, a socket interface and so on. You should define standard object interfaces for these basic classes, and then you provide physical adaptations for those, and thus gain device-independence.

We may have different thoughts about what the object model does. My common object model only defines how classes and interfaces work. It doesn't dictate the contents of interfaces. Just like you said, I do have interfaces for files, directories, sockets, and devices, but these are unified under the common object model.

The way I do it in my OS, is that I have a module that implement handles (creating, dereferencing, deleting), but that doesn't know the purpose of the object it protects. Then I have a list of handle-types which are associated with each handle. When a handle is dereferenced, the handle type is passed and checked, which makes sure that handles are of the correct type.

The object manager in my OS seems to do exactly what your OS's handle module does. My object manager is responsible for reference counting, security, and handle management, but it doesn't care about the contents of the objects.

[linguofreak]

Hi,

The reason why files are used and not objects is that because if you add new objects to the system, every program that doesn't know about this new type needs to be updated to be able to use them, you dont have that problem to the same extent with files. Have you thought about that?

That's not quite right.

In systems that use "files" for device driver interfaces you end up with a whole bunch of problems (caused by lacking functionality) because devices aren't files at all and need to support device specific things that are more than just transferring data. This is followed by IOCTLs as a work-around for the missing functionality problem, followed by "every program that doesn't know about this new set of IOCTLs needs to be updated to be able to use them".

One thing I've mused about at times is using directories for device driver interfaces. Each device file on a traditional Unix system gets turned into a directory holding at least a "ctl" file and a "data" file. The "data" file takes over the role of the original device file, and instead of using ioctl(), processes write command strings to the "ctl" file. So you might have /dev/fb0/ctl and /dev/fb0/data (or, for back-compatibility with existing Unixes /dev/fb0dir/ctl and /dev/fb0dir/data, with /dev/fb0 being a symbolic link to /dev/fb0dir/data, and ioctl() being a userspace libc function that opens and writes to /dev/fb0dir/ctl).

[Marionumber1]

One thing I've mused about at times is using directories for device driver interfaces. Each device file on a traditional Unix system gets turned into a directory holding at least a "ctl" file and a "data" file. The "data" file takes over the role of the original device file, and instead of using ioctl(), processes write command strings to the "ctl" file. So you might have /dev/fb0/ctl and /dev/fb0/data (or, for back-compatibility with existing Unixes /dev/fb0dir/ctl and /dev/fb0dir/data, with /dev/fb0 being a symbolic link to /dev/fb0dir/data, and ioctl() being a userspace libc function that opens and writes to /dev/fb0dir/ctl).

Plan 9 does something similar to your idea. It doesn't actually have an ioctl() syscall, instead it uses control files:

Unlike most other operating systems, Plan 9 does not provide special application programming interfaces (such as Berkeley sockets, X resources or ioctl system calls) to access devices. Instead, Plan 9 device drivers implement their control interface as a file system, so that the hardware can be accessed by the ordinary file input/output operations read and write.

[Marionumber1]

Another thing I was recently thinking about was how component model classes are registered. If they're registered using human-readable names (such as File, Device, Canvas, Window), it's easier for programmers to use, but that opens up the possibility of name collisions. If I use GUIDs for each class, the name collision problem would be avoided, but the name means nothing to programmers, making it harder to use the component model. What are your opinions on this?

[Jezze]

If it means that you as a programmer needs to supply a GUID in order to access a certain resource I'd would say no way. I somehow think that it should be possible to fix collisions using namespaces somehow so that they wouldnt conflict with eachother and if you get a conflict when trying to add a new resource it would fail similar to what would happen with a a "file already exists" or possibly it could replace / shadow the resource which could be kinda neat because then you could create specialized resources that replaces more generic ones. A third option would be to merge the resources. I dont really know what would be the best option for you.

[Marionumber1]

If it means that you as a programmer needs to supply a GUID in order to access a certain resource I'd would say no way. I somehow think that it should be possible to fix collisions using namespaces somehow so that they wouldnt conflict with eachother and if you get a conflict when trying to add a new resource it would fail similar to what would happen with a a "file already exists" or possibly it could replace / shadow the resource which could be kinda neat because then you could create specialized resources that replaces more generic ones. A third option would be to merge the resources. I dont really know what would be the best option for you.

Those ideas could work very well. I'm thinking of structuring my classes in a hierarchial namespace, sort of like how .NET does it. It could work somewhat like this:

Code: Select all

Audio
    Audio.MP3
    Audio.WAV
    Audio.WMA  
Image
    Image.BMP
    Image.JPG
    Image.PNG
System
    System.Graphics
        System.Graphics.Canvas
        System.Graphics.Window
    System.IO
        System.IO.File
        System.IO.Section
    System.IPC
        System.IPC.Queue
        System.IPC.Pipe
        System.IPC.Socket
        System.IPC.Semaphore
        System.IPC.Mutex
        System.IPC.RWLock
        System.IPC.Timer

[jbanes]

Those ideas could work very well. I'm thinking of structuring my classes in a hierarchial namespace, sort of like how .NET does it. It could work somewhat like this:

Code: Select all

...
System
    System.Graphics
        System.Graphics.Canvas
        System.Graphics.Window
    System.IO
        System.IO.File
...

OOP had its day. While it helped everyone better structure their thinking, it's not a good concept to structure a modern system around. The couplings are too strong for the system to hold up under any level of complexity.

Unless you like visiting endless amounts of pain on engineers, that is. In that case go right ahead.

[AndrewAPrice]

If you go the managed language route, you could make "everything an object" and freely pass interfaces between processes.