my os design for the moment

[h0bby1]

i started to get into developing this os for about one year on and off, and been developing various related thing before, i used to make light weighted bootable linux with audio and accelerated opengl in about 4Mo bootable image, and did few test with simple boot sectors who can initialize audio card and play sinus with keyboard key, and been reading pc hardware related specifications for a while, and i have been programming many kind of things on linux and windows, but i'm not sure if i'm doing everything right, i didn't plan to go that far with it, but as it goes well, i just can't stop adding more and more things and see how far i can get it going

my original goal with the os is mostly to provide simple framework to launch bootable application, that would be more or less focused on video and multimedia, streaming, encoding, and eventually 3D and graphic/audio application in a simple manner to avoid all the complexity of modern os to see where it can get

for now i didn't really plan to support multicore, the first thing i developed in it is my own ABI, it has the feature that it can be loaded entirely 'in place', including dynamic linking and relocation, in sort that i can have some dynamically linked code included in the kernel, and doing the relocation/import and getting the exported location without allocating anything

i use the segmented memory model, and i use the same memory zone for data and code segment, so i can jump to any data and execute it , and i use a tool to convert dll or so file to this binary format including the relocation and import/export and eventually compression, i can link object made from dll or so together generally it works well

there is also a second binary loader made in C that is implemented in one of the file included and loaded by the kernel at boot time, this loader is more evolved and load only data/code sections in a dynamically allocated memory location, so technically i think it could handle as well to load the code section in a specific code segment added in the gdt with the permission, and do the relocation in the virtual address space

i plan also to have two version of the libc, one version of the libc for kernel space that is to be used for all the base system part, and another version that would not include any kernel call and would be loaded in application space for each application

the applications loaded in application space would not need to import lot of thing except libc function, and i would implement probably a message queue in application space that the system would poll to execute commands, and it can use a system that can define tree based structure that can be seen as hybrid between vfs and registry base to define data for more complex operation

all the rendering interface already use this tree system to define rendering structure, and i think it should be rather easy to also put the functions to manipulate graphic related tree structure into application space for that the application can manipulate the rendering engine easily, i already have simple function to manipulate the tree structure such as

win_id = gfx_create_render_container (100,100,400,400);

gfx_container_add_image (win_id,10,10,"isodisk","/system/imgs/image.png");
txt_list_id = gfx_container_add_text_list (win_id,text_pos_x,text_pos_y,"Flama",12,12);
text_id = gfx_container_add_text_to_list (win_id,txt_list_id,n,0,text);
gfx_render_containers_set_scroll_y (win_id,0,scroll_Y);


for example, and it just add some node into the container tree structure, which could be allocated in application space, like this i don't even really need compositing or having window/Container specific backbuffer and everything is directly rendered on the framebuffer from the tree information in each container, as application never use directly the rendering interface, there is not real point in having windows specific backbuffer

the memory allocation system is based on defining memory area into which the application allocate zones, so i could set up memory area specific for all application space, with the segmented memory model it make it easy to translate virtual address to physical, it's mostly only about subtracting/adding the ofset of the start of the segment, and memory area could be defined with a virtual ofset that match the start of the segment to provide application space virtual address

i don't plan on doing it massivly smp and multi threaded, but i still plan to be able to run application also in application space only communicating with the system with a message queue in application space that the system would poll , and also maintaining a window/container list in application space and the function to manipulate them, the application space would mostly contain the libC, a lib to manipulate message queue and graphic rendering interface, and having a system for that the kernel can call application space function to handle event

the way i see it, interupt would only need to be handled by kernel space code, mostly on a single core, and i would more either do a software routing of events that are linked to an application rather than having interupt handler in application space, but i don't know if i might be thinking it in a wrong way

otherwise for actual kernel space and device management, it's all based on a bus manager, that maintain a list of device and drivers, the bus manager load bus drivers like pci/usb drivers who are responsible to manage device enumeration and store the devices information in a device list accessible from the bus manager, it use the same tree system that is close to registry base or vfs, and then the bus manager do the drivers/device match in a generic manner from the bus manager, either based on class or vendor id/product id

the bus manager is also responsible for registering and dispatching device related event, the application register an event handler for a particular type of device like mouse or joystick to the bus manager, and the device drivers like usb hid parse the bus manager event list and call the registered event handlers

the bus driver (pci and usb) is responsible for loading and initializing device drivers for devices present on that particular bus, the bus manager hold the list of drivers that it load from a directory for each bus, and extract the class/product id the driver can handle from exported symbols, it doesn't even have to do a full loading of the drivers to get a pointer to the exported symbols that hold those information

device drivers can register some stream objects linked to the device in the bus manager for things like audio device or network card, to be able to find a stream associated with a particular device type and then being able to read/send data to it in various way (sync/async/continuous polling, initiated by device or host etc)

i plan to do in sort that application just have to define what the os must do with hardware request, like let say an application want to play an mp3, it could just build a stream chain descriptor and link it to the audio device stream for that the os can know what to do when it recieve an interuption without application interaction, same with filters/effects, or networking with packetizers or having a system to have custom application handling of event if needed for particular purpose

[sortie]

Hi, I see a lot of text that's poorly formatted and full of spelling errors. Would you mind fixing it up a bit so I actually want to read it?

[h0bby1]

Hi, I see a lot of text that's poorly formatted and full of spelling errors. Would you mind fixing it up a bit so I actually want to read it?

oki, i tried to correct spelling and improve formatting a bit

[bluemoon]

Is it just me, I failed to see any design but a list of component names and some implementation detail, without goals or how things works.

[h0bby1]

Is it just me, I failed to see any design but a list of component names and some implementation detail, without goals or how things works.

it's a global outlook, globally the component are, for the kernel ( it explain how they interact with each other):

bus manager
bus drivers
device drivers

then there is the graphic base which is the part with the container, it include font mannagement, image loading, and windows/container

actual renderer are in the device drivers, the system init routine check for a graphic device in the bus manager, and then initialize the graphic base renderer with function exported from the specified device driver i have made some diagrams to explain a bit i attached it in this post normally

it doesn't detail everything, just global outlook, and also how i mannage memory and exectuable module

[rdos]

I think the layering is wrong. For instance, PCI is the basis for many other devices like USB, network cards and basically everything in modern computers, and thus cannot be at the same level as USB.

[Combuster]

PCI is the basis for many other devices like USB

Basically, no. Just remember that the vast majority of computers have USB but no PCI. The same goes to a lesser extent for networking and graphics.

A computer is typically built as a hierarchy of buses. USB is a comparatively slow bus, so you can typically stick it under a faster bus, like PCI, but just as well directly onto the system bus.

[h0bby1]

the design for low level is strongly based on the bus manager, i took the idea from haiku as i think it has something similar inside

there is no hierarchy in bus like that, a bus drivers is just an abstraction to put together a list of device and drivers , pci bus just allow to retrieve the setting to initalize the usb controller, and the usb bus drivers is initalized from the usb controlller (ohci/uhci) pci device driver code, and it initalize the usb bus drivers with the bus id/dev id of the ohci/uhci device, the usb bus drivers can then find the drivers of that device throught the standard bus mannager interface who hold the list of device and drivers in a geneic manner for all buses, and import the symbol used to manipulate it, but it doesn't depend on the pci bus , it interact with the usb controller transfer queues and port status directly througth symbols exported from the pci drivers of the usb controller, but they are not functions that are part of the api that the pci bus use to manipulate the device as a pci device, they are another api used to manipulate the controller as an usb controller that the usb bus drivers use

this is from the ohci pci device drive code, it create a new usb bus drivers and initialize it with the bus/dev id of the usb controller

Code: Select all

	usb_bus_drv_id		=	bus_manager_load_bus_driver	("ramdisk","/usb_bus.tpo");

	if(usb_bus_drv_id==0xFFFFFFFF)
	{
		kernel_log		(kernel_log_id,"could not load usb bus driver \n");
		return 0;
	}

	
	kernel_log		(kernel_log_id,"usb bus driver loaded ");
	writeint		(usb_bus_drv_id,16);
	writestr		("\n");

	_config_bus_manager_id			=	usb_bus_drv_id;

	bus_manager_set_bus_driver_ctrler	(usb_bus_drv_id,_config_bus_id,_config_dev_id);

then it use those values (bus_drv->controller_bus_id,bus_drv->controller_dev_id) in the usb bus to get an handle to the driver module who contain the functions to manipulate the controller

Code: Select all

kernel_log	(kernel_log_id,"scanning usb bus \n");

	if(bus_manager_get_device_driver		(bus_drv->controller_bus_id,bus_drv->controller_dev_id,&usb_controller_drv)==0xFFFFFFFF)
	{
		kernel_log	(kernel_log_id,"could not find usb controller driver \n");
	}

	kernel_log	(kernel_log_id,"controller driver:'");
	writestr	(usb_controller_drv->name);
	writestr	("' bus id:");
	writeint	(bus_drv->controller_bus_id,16);
	writestr	(" dev id:");
	writeint	(bus_drv->controller_dev_id,16);
	writestr	("\n");


	controller_usb_hubports			=get_tpo_mod_exp_addr_name			(usb_controller_drv,"usb_hubports");
	controller_usb_detect_dev		=get_tpo_mod_exp_addr_name			(usb_controller_drv,"usb_detect_dev");
	controller_usb_portstatus		=get_tpo_mod_exp_addr_name			(usb_controller_drv,"usb_portstatus");
	controller_usb_ctrl_transfer	=get_tpo_mod_exp_addr_name			(usb_controller_drv,"usb_ctrl_transfer");
	controller_usb_intr_transfer	=get_tpo_mod_exp_addr_name			(usb_controller_drv,"usb_intr_transfer");

i do something similar with the rendering drives, bus device divers have the api used by the specific bus driver to initialize the device, and then another api that can be retrieved from the root bus mannager into the same drivers object to manipulate the controller, probably the same system will also be used to mannage ATA controller

if there would be some function specific to the bus the device is attached on to manipulate it, the device driver can still communicate with the root bus mannager and manipulate the bus it is attached to from there

the function that are bus specific are handled in the api for the bus the device is programmed for, and only manipulated from the bus drivers who scan the device and load the drivers and do the base initializitation of the device using bus specific function, from there the device should be able to be manipulated using bus independant functions

but normally the goal of the system is to make also the bus interface transparent and that the devices can be manipulated with a generic interface no matter the bus it is attached to, and to avoid bus specific command to manipulate a device, so the bus hierarchy is supposed to be made transparent from the root bus mannager api, it's just manipulated using either device type and index for that device type, or bus/dev ids, and then using function exported from the device drivers attached to it to manipulate the controller/device in more advanced manner

for device who are not initialized from a bus drivers, i would create a bus drivers as a root sys bus, who scan all device spresent on the mother board and load a drivers module for them, the device and drivers are stored in the bus mannager in a generic manner for all buses, the drivers for a particular device can then be found using some function to parse a bus from the root bus mannager api, and then function to manipulate the controller are specific to the type of device, but the code of the pci bus is not needed to program the function of the controller once it is intialized

interupt event are also mannaged throught the root bus mannager api, device drivers can notice the bus mannager of an event, either throught their device type ( a match is made either from class ids or other device specific infos by the drivers for the particular bus), and application also manipulate device and register event handlers for them also through the root bus manager api

but the graph do not show all part of the os, it's not very detailed either i should write more doc about it somewhere

[rdos]

It's true that PCI is only involved in the initialization part of these devices, typically getting some base address and possibly an IRQ (with the help of ACPI). But, OTOH, PCI isn't really a device either, rather an interface that can enumerate devices.

[h0bby1]

It's true that PCI is only involved in the initialization part of these devices, typically getting some base address and possibly an IRQ (with the help of ACPI). But, OTOH, PCI isn't really a device either, rather an interface that can enumerate devices.

there is a pci bus bridge that is an actual device on the computer, and internally it does more than just fetching information from the config space, and it has also some flags regarding how the bus should behave with the device, regarding bus master, and dma settings, it's a device but lot of its function happen internally at hardware level, but it can still be considered as a device as it has a command/data register, but it's mostly manipulated by the bus drivers, and higher level function should use the generic bus manager interface if they have something to do with the bus, and device drivers api to access and program devices

for the usb bus as well, the 'root hub' to enumerate device is still a device that need to be programed specifically to enumate device and initialize them

'bus root' device can be programmed to deal with latency, scheduling or other things in the absolute

bus controller have to be considered as a device, but a device that is programmed throught a bus driver and not a device driver