Trying to set up a stack segment with limits and failing

[iansjack]

Paging and the flat memory model provide NO PROTECTION AT ALL within an application, or between device drivers in a kernel.

Of course it does. Mark a code page as read only and the program can't overwrite it. Use guard pages and you protect against stack overflow.

[rdos]

Paging and the flat memory model provide NO PROTECTION AT ALL within an application, or between device drivers in a kernel.

Of course it does. Mark a code page as read only and the program can't overwrite it. Use guard pages and you protect against stack overflow.

That's pretty useless. The problem is not that you might overwrite code, but that you start to execute code you shouldn't execute. Guard pages might work for the stack, but only if you don't attempt to grow it more than 4k. You also forgot about the biggest problem: malloc/free. In the flat kernel, drivers use malloc to allocate global addresses, and when they happen to overwrite them, they overwrite unrelated stuff, like the scheduling lists, task blocks or some vital device driver data. I have yet to see an OS that provides local malloc per device driver, or some other more efficient means of protecting drivers and kernel from each others.

After all, the micro kernel approach is aimed to solve these issues, but at the expense of TLB flushes and address space switches.

[Demindiro]

The problem is not that you might overwrite code, but that you start to execute code you shouldn't execute.

That's what the NX bit is for.

Guard pages might work for the stack, but only if you don't attempt to grow it more than 4k.

Stack probes protect against that by touching every 4K page.

In the flat kernel, drivers use malloc to allocate global addresses, and when they happen to overwrite them, they overwrite unrelated stuff

Segments do not solve that problem. As you mention, microkernels partially address this, but what is better is the use of a language with a proper type system (i.e. not C).

I have yet to see an OS that provides local malloc per device driver, or some other more efficient means of protecting drivers and kernel from each others.

A popular "efficient means" is Rust, which outright rule out the class of bug you mention, the only requirement for prevention being a small set of thoroughly reviewed code wrapped in "unsafe" blocks. If those unsafe blocks are proven safe, then "Safe Rust" statically cannot cause memory bugs.
Though as I mentioned before, devices frequently do DMA, which do not care one bit about segmentation or paging.

After all, the micro kernel approach is aimed to solve these issues, but at the expense of TLB flushes and address space switches.

ASID (or PCID in x86 terms) significantly reduce the amount of TLB flushing necessary by allowing multiple address spaces to exist in the TLB simultaneously.

These same arguments keep being rehashed over and over again. Why is it you seem to always forget about them? Or shall we have another derailed thread?

[nullplan]

I have yet to see an OS that provides local malloc per device driver, or some other more efficient means of protecting drivers and kernel from each others.

You keep repeating this talking point and I am bloody well tired of it. In practice, there is no protecting the kernel from the drivers or vice versa, because each driver can DMA over any "protected" part of the address space, entirely circumventing all CPU-based protection mechanisms. For this reason, any discussion on the relative merits of segmentation vs. paging is entirely academic.

The fact that segmentation is an x86-only thing, whereas paging is available on everything in some form at least, makes it entirely unsuitable for any OS even attempting portability.

[rdos]

In practice, there is no protecting the kernel from the drivers or vice versa, because each driver can DMA over any "protected" part of the address space, entirely circumventing all CPU-based protection mechanisms. For this reason, any discussion on the relative merits of segmentation vs. paging is entirely academic.

Just because things like PCI devices can write to any physical location in memory without consulting the OS, doesn't mean protecting drivers from each others has no benefit. Protection is not a black-and-white concept that you either have or not have. It needs to be layered in an appropriate way. The most vulnerable part is physical memory, so access to it should be restricted to a few trusted modules. There are also smart algorithms for constructing in-memory schedules that many modern PCI devices work with that makes the probability of misusing physical memory lower.

At the next level is linear memory. Paging dictates which physical memory can be access from which linear address. If some physical memory is not mapped in linear memory, it cannot be accessed. By having many different address spaces you can further limit access to physical memory, and provide more secure environments. If your physical memory manager is no good, then paging cannot save you. About the worse practice around is to map all physical memory in linear memory, as this in one step makes all physical memory directy available.

Segmentation operates on top of linear memory (and on top of paging, if used). Segmentation further limits which linear memory is available in a given context. As noted above, what is not mapped, cannot be accessed. In my design, each driver has it's own code and data selector, and cannot access other code or memory other than by syscalls or far pointers.

In fact, using your reasoning, paging is just an academic thing with no merit, so why not write your OS without paging (or with identity mapping)? After all, DMA and PCI devices can write to any physical address, so why bother with paging?

[Demindiro]

In fact, using your reasoning, paging is just an academic thing with no merit, so why not write your OS without paging (or with identity mapping)? After all, DMA and PCI devices can write to any physical address, so why bother with paging?

User processes typically don't have direct access to devices, so those issues do not apply for regular applications.

Those issues do apply for drivers and it indeed makes perfect sense to not bother with paging for drivers. See: every monolithic kernel.