Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
I am trying to implement cooperative multitasking into my project. My project isn't a real OS. I am using FreeDOS and want to write a GUI for it. As I can't modify the interupts, FreeDOS is using, I am implementing cooperative multitasking. Every process should return to my scheduler, when it meets an interupt. So while loading the binary, I am replacing every "INT" with a "RET" and during execution, I am calling the memory address, where I've loaded the binary to and adding the current IP to it. Like this (pseudocode):
load {
do {
opcode=get_byte_from_file();
if (opcode=INT) {
opcode=RET;
add_interupt(); // Save interupt
}
program+=opcode;
} while (not_eof);
}
execute {
pointer=program+position;
asm {
// load registers
call [pointer]
// save registers
}
current_int=get_next_interupt();
position=current_int.position;
if current_int.interupt=33 {
// catch DOS interupts
}
}
Everytime "execute" is called, it executes a part of the process, until it encounters a "RET" (which was an "INT" before). Since I have saved the interupts with their positions into an array, we can catch those interupts and execute them. This sorta works and I can execute some very simple programs.
However, there are a bunch of problems with this implementation:
1. The whole thing becomes useless, when there is a real "RET" in the code.
2. It won't return to "execute", when the process is pushing values onto the stack.
My question is: How would you implement cooperative multitasking? Is there another way to execute processes and catch interupts, without modifying interupt vectors?
First, to get going specifically for multitasking you need to:
- Create a static kernel with Ring 0 kernel, Ring 3 kernel, system and user level code, and all the programs you would ever want to run, into the one same kernel binary image. You will boot that image loading it at once. You won't be able to scale without an initially static image with kernel and ALL applications, without having any other prior advanced subsystems readily implemented, so it's the best way to start in those advanced subsystems that I can come up with for a novice OS developer without prior code of their own. This is how we will solve the problem of novices to reach an usable implementation in those areas.
- Implement a physical memory manager, practically identical to the memory allocator for virtual address spaces. Initially we won't use paging but instead we will start by implementing a physical memory manager that allows us to treat physical memory transparently as if it was one more virtual address space. It will fragment as much as a virtual space when allocating and deallocating memory blocks, so it will largely be the same.
- Implement paging. We will need paging to evaluate how to implement memory protection, but it won't be possible and won't make sense before having the previous points ready, built at compile time for a boot start try.
- Remember to create an image with separate programs and stuff separated in page boundaries and with a default hard-coded paging map, in fact a full PCI bus map, multitasking structures per each contained program (will practically be already running probably requiring minimum setup), VGA map, floppy CHS to LBA map, everything you need to run if you already had a finished system and were waking up from an hibernation image. in raw source code to compile or assemble. It will allow you to just load your test kernel as if it was an hibernation image. Probably even Windows is calculated in such a way that when you load all system DLLs and components, a hard-coded paging map is retrieved up to the point where physical memory is installed, but for a start we can create a full test paging map just to determine how to use it initially without errors, without bogus functions, fully by hand, until we come up with a way to detect installed RAM and build paging structures dynamically.
- Once you have all ready and can terminate programs, you will be able to strip the initial in-memory image from unused chunks and keep running programs normally from disk. Then you can have your kernel tell you FROM THE START with truly good messages that tell you what code portions have been unloaded, and from there you will be able to know what not to include in your new final kernel, but back up your previous code for educational value. After all, your applications will be embedded in a source code subdirectory structure and several multitasking structures initially hard-coded. You would mostly need to remove one single "main.asm" or "main.c" include file to remove all of your already-loaded-at-compile-time test programs by removing a single test include file for a huge branch of test code that wouldn't be really useful for an end system (with PE EXE, NE EXE, LE EXE, MZ EXE, COM, AOUT, COFF, Mach-O or ELF structure). Anyway it will also make you capable of loading everything you need at once for example for computers where you don't know how to control newer hardware but keep running nonetheless.
________________________________________
________________________________________
________________________________________
________________________________________
You could make that Alt+Tab return to your kernel console via a task list to choose from.
Each task could have its own console or graphics mode.
You could add a flag that tells which process is currently focused for each thing, for graphics, keyboard, etc. If the current application isn't currently focused, it can continue running in the background but without trying or being able to alter the screen, unless it's intended to do so.
If you give each program its own text or graphical console, then that will be the only window running and you will need to learn how to implement programs visually running simultaneously in the same console session.
For cooperative multitasking, it's the program itself that needs to call a kernel function to return control to it. While it doesn't do that, it will keep monopolizing the CPU as no other program will be running and the CPU will be frozen until you return control to it.
From there, it's easy to implement automatic preemptive multitasking, ensuring that every time that an interrupt service is called, or every time a system call is made, those countdowns will deplete the chances to keep running for the current program. You can simply assign a token countdown for a program. You can create an interrupt token countdown and an API call token countdown. After several interrupts of any kind, not just the timer(s), while running an application, and more importantly after exhausting the API token countdown, you can decide how to switch to another program when each of those token types are exhausted. You could renew the exhausted count and then switch again when the other token types are exhausted, as if they were coins to play a game.
It's easy to switch tasks in this way because after all you are calling the kernel directly or indirectly, and your program has a set number of chances to call the kernel and experience hardware interrupts before needing to switch by the kernel. After all you are calling the kernel and that's where you can make every API function in your kernel library to check if it's time to switch tasks.
If the interrupt data corresponds to your program, then keep running it. If it belongs to another program, then switch to it to service the interrupt in the context where it's being used. You can interrupt immediately and return immediately to the currently executing programs, and then run the background processes according to their importance. The count of interrupt tokens and API call tokens will be the ones that will control mostly the priority of a program. The higher those two token counts, the higher the priority of a program.
You can have preemptive multitasking to ensure that no program will freeze the system.
But you can also keep intact the cooperative multitasking APIs and functionality.
By doing so, your program can also voluntarily tell the kernel to run another program in addition to be switched after exhausting its allocated chances to run continuously based on coherent events.
By keeping and using preemptive and cooperative multitasking in the same tasking system, we can improve the performance of the system even more.
____________________________________________________
____________________________________________________
____________________________________________________
____________________________________________________
You can implement multithreading to begin with. Then you can experiment with cooperative and preemptive multitasking from a program.
Doing that the kernel is the same. In that context, programs are considered extensions to the kernel run-time. But for that you mostly need virtual memory with paging or code relocations, for which you need to understand paging and also CPU emulation to fix the literal immediate addresses in the binary image at load time.
That's why it's better to start learning with a single multithreaded program that contains everything you need to do in a given moment.
You could hack together a kernel that contains all basic programs in a same image for easily multitasking them as you would have loaded them to their right final addresses just to test with the structure that your proper system will have when you learn to relocate or fix virtual addresses, without really needing paging but physical memory management will be necessary just like for a virtual space.
When you are able to fix the addresses of a binary that hasn't been relocated to its proper end base address, then you will be able to implement proper full multitasking, but in the meantime you will have to learn and test with a single monolithic kernel file with all the programs you want to use embedded into one same binary with the memory layout structure you will really be implementing.
That's why I'm trying to add debug messages logged to a debug-only ATA hard disk to learn how paging in other system works, like CWSDPMI for DOS, MenuetOS, VisOpSys, ReactOS or Linux. It will be much faster to learn from them first hand.
Remember to create an image with separate programs and stuff separated in page boundaries and with a default hard-coded paging map, in raw source code.
__________
It will probably be a long time when I have many programs that I can embed in the kernel, to recompile the system to choose the programs I plan to use according to installed RAM, for example music player, certain driver images now expanded in the kernel memory image, web browser, video player, text editor, windowed file manager, calculator, assembler, compiler, eMule, BitTorrent, image editor, file archiver, command line consoles, web server, game emulators, clock, screen recorder, hex viewer, PC emulator, process viewer, TV program expanded from binary to be included in RAM (probably with manual relocations, not just including source code but also practice expanding and relocating executables to RAM at compile time to adequate places)...
Since all or part of the system RAM will be accessible at low level directly, I will be able to test snippets searching manually for a big enough free chunk of memory and the tools present on RAM, which probably wouldn't be a good idea to destroy from memory when they stop running, just look a way to reuse them.
Last edited by ~ on Mon Jul 31, 2017 4:17 pm, edited 1 time in total.
Postmann wrote:My question is: How would you implement cooperative multitasking?
I wouldn't. Preemptive multitasking is much better.
If I really needed cooperative multitasking for native GUI programs, I'd take Microsoft's idea and design the API so that programs have to repeatedly call a particular function for the UI to work. This function, aside from being necessary for the UI, would also interact with the scheduler in order to allow multitasking.
But that only works with native GUI programs. For DOS programs, I'd take Microsoft's idea again and do it as a DOS extender. By running the DOS program in ring 3 with virtual 8086 mode, you can easily trap all sorts of events to allow task switches. (Depending on the events you decide to trap, you might end up doing preemptive multitasking without realizing it.)
Postmann wrote:Is there another way to execute processes and catch interupts, without modifying interupt vectors?
Cooperative multitasking is necessary to optimize the system execution, along with preemptive multitasking, so that programs can indicate the kernel that they want to stop running a thread or the main process and run another process instead if they aren't going to do anything, for example if there are no new events. Even better than just trying idling with HLT while a time slice finishes as we will actually switch to a program that will likely do something immediately. Does more than just wasting time slices.
Also useful to have more control on keeping or decreasing the priority of the program from itself.
The next process to run when a program cooperatively returns control to the CPU in a preemptive system, is chosen by the kernel.
The kernel could detect that a process doesn't have new events or interrupts and avoid waking it up, but there are countless cases where voluntarily returning control of the CPU to the kernel cooperatively in addition with the automatic preemptive base would definitely benefit the system.
If there are lots of programs that would be suddenly idling but that would have free time slices, they all would collectively work better if they all returned CPU control as soon as they know that they will no longer do something in this time slice, for other process to employ better.
Last edited by ~ on Mon Jul 31, 2017 6:23 pm, edited 1 time in total.
