Separate Stack Segment in Protected mode?

[iansjack]

I can think of better arguments in favour of long mode than a 17-year-old research paper.

[Demindiro]

Perhaps, but also a good argument for why you don't want to use long mode.

I think the performance benefit alone is enough to negate any arguable downsides.

[rdos]

Perhaps, but also a good argument for why you don't want to use long mode.

I think the performance benefit alone is enough to negate any arguable downsides.

Not related to long mode. It was primarily a compiler issue.

[rdos]

I can think of better arguments in favour of long mode than a 17-year-old research paper.

Some things never change. Running an OS kernel without effective protection mechanisms in place is insane. I don't count paging as an effective protection mechanism since it has poor granularity and no limit checking. A decent micro-kernel design *might* be acceptable, but neither Windows nor Linux use that design. The problem becomes even worse when people decide to map all physical memory in the address space, and pack code & data in the executables.

[Demindiro]

Perhaps, but also a good argument for why you don't want to use long mode.

I think the performance benefit alone is enough to negate any arguable downsides.

Not related to long mode. It was primarily a compiler issue.

The long mode version is 25% faster than the protected mode version.

[rdos]

The long mode version is 25% faster than the protected mode version.

That's not important. The important thing is that flat kernels never become bug-free and poorly written drivers easily can bring down the entire OS since the environment is completely unprotected.

Actually, you need to switch to interpreted code to achieve decent memory protection, and then we are talking about many times slower code.

[nullplan]

That's not important. The important thing is that flat kernels never become bug-free and poorly written drivers easily can bring down the entire OS since the environment is completely unprotected.

Andy Tanenbaum would like a word with you. I am highly suspicious of anything that claims bug-freedom, and anything that claims it can somehow insulate itself from the effects of poor drivers. Badly written drivers can make the hardware overwrite your kernel, whatever protection measures you deploy. Or melt the hardware or something.

Actually, I would like to know your threat model. What is it you seek protection from?

[rdos]

That's not important. The important thing is that flat kernels never become bug-free and poorly written drivers easily can bring down the entire OS since the environment is completely unprotected.

Andy Tanenbaum would like a word with you. I am highly suspicious of anything that claims bug-freedom, and anything that claims it can somehow insulate itself from the effects of poor drivers. Badly written drivers can make the hardware overwrite your kernel, whatever protection measures you deploy. Or melt the hardware or something.

Actually, I would like to know your threat model. What is it you seek protection from?

Threat model? It's really easy. The less a piece of code has direct access to the less likely it is corrupt some vital data in the kernel. So, by mapping all physical memory in the linear address space, and by using a flat memory model, you basically give every line of kernel & driver code the ability to corrupt vital kernel data and physical memory that is protected by being mapped in server processes or that is private to PCI devices. It can't get any worse than that.

If you know Murhpy's law you also know that if something can go wrong, then it will, and at the worst possible time.

As a counter example, RDOS drivers have a private code & data segment with exact limits. As long as it accesses using cs and ds, it never can corrupt the kernel. I don't have physical memory mapped in the address space, and so a driver needs to call a kernel mapping function to map a physical address to a linear and only then can it corrupt it. All pointers in the user space & kernel API are 48-bit and must be passed in selectors & registers. The user space flat selector has a 3G limit, and so user space can never corrupt kernel by passing bad pointers to kernel since kernel data is mapped above 3G. Actually, I have no pointer validation in syscalls since this is not necessary.

[iansjack]

So what’s wrong with using the protection inherent in paging?

[Demindiro]

RDOS drivers have a private code & data segment with exact limits. As long as it accesses using cs and ds, it never can corrupt the kernel..

It can by instructing hardware (e.g. xHCI) to write to specific physical addresses, and hardware does not care about either segmentation or paging.

There are IOMMUs of course, but AFAIK not all platforms/hardware have one/support it and it also needs some sort of driver too.

[rdos]

RDOS drivers have a private code & data segment with exact limits. As long as it accesses using cs and ds, it never can corrupt the kernel..

It can by instructing hardware (e.g. xHCI) to write to specific physical addresses, and hardware does not care about either segmentation or paging.

There are IOMMUs of course, but AFAIK not all platforms/hardware have one/support it and it also needs some sort of driver too.

Actually, no. All my USB drivers use a specialized kernel API that makes linear to physical (and the reverse) translations easy, and also localizes them to only a few physical pages. Thus, the XHCI device will never get bad physical addresses as long as drivers use the appropriate API only. When it comes to mapping transfer buffers, the driver gets a 48-bit pointer, gets it's base and then use a kernel API to get the physical address. Thus, the driver will never work with physical address as parameters, rather extract them from known-to-be valid pointers using a known to be reliable kernel API.

Of course, if a malicious driver decides it want to corrupt something, then of course it can. However, this is not the point. The point is accidental corruption through writing bad code and primarily by using bad pointers.

[rdos]

So what’s wrong with using the protection inherent in paging?

It only works between application, server processes, and kernel, provided all parameters are properly validated. It doesn't protect kernel from buggy drivers since in a flat memory model, every piece of code can access everything in kernel. Paging cannot solve that issue. Only segmentation can solve it.

However, if you break out drivers from the kernel and put them in isolated server processes that run in user mode, then you can provide good protection between the driver and the kernel even if the driver use a flat memory model. However, this costs a lot more than segmentation, and no big OS that I know of use this method.

[iansjack]

If you can arrange for different processes, or drivers, to use different segments why can’t you arrange for them to use different page tables?

In reality, you can’t protect against rogue code running in supervisor mode. It can create any segments or page mappings that it wants to. I think the protection you think you get from segmentation is totally illusory.

[rdos]

If you can arrange for different processes, or drivers, to use different segments why can’t you arrange for them to use different page tables?

Because you need to shoot down the TLB every time you switch to another function. This is much more expensive than loading a selector.

In reality, you can’t protect against rogue code running in supervisor mode. It can create any segments or page mappings that it wants to. I think the protection you think you get from segmentation is totally illusory.

I know it is not illusory. I use applications that run in flat memory model, and so can compare with the kernel that run in segmented mode. Practically all pointer bugs in kernel create faults at the location of the problem, while in the application, it can frequently be an effect of some other code that corrupted stuff. Paging sometimes catch the issues, but only when I use the debug library which allocates 4k for every new, initialize it to known values, and check for overwrites when freeing it. I cannot use this in production releases since it consume too much memory and the validation is slow.

Also, most of the kernel is written in assembly while most of the application is in C++. Still, there are many more hard-to-find bugs in the application than in the kernel. There are basically no sudden death occurances in kernel at all. Segmentation basically catches all bugs.

You might think of it like this:
Say you have a pointer in a driver. In a RDOS driver, it will be a 32-bit offset from the drivers data segment. In long mode, it will be a 64-bit linear address. Now say something corrupts the pointer. In the RDOS driver, random stuff in the drivers data segment could become corrupted by using the pointer, but any pointer that is outside of the data segment will fault, so the pointer never can corrupt kernel data. The same scenario in long mode can lead to corruption of physical memory, vital kernel data, application data in another process, and even PCI BAR data.

[Schol-R-LEA]

We've gone over this many, many times before, rdos. Segmentation is not, and has never been, a protection mechanism, any more than paging is. While the protection mechanisms work together with them, it is not the protection mechanism, nor does it provide any more or any less protection than paging does.

The same scenario in long mode can lead to corruption of physical memory, vital kernel data, application data in another process, and even PCI BAR data.

That's simply not true, or rather, the claim that segmentation would prevent it is incorrect. Supervisor-mode pages have exactly the same degree of protection as supervisor-mode segments - a wild userland pointer to a supervisor data page is still going to be blocked by the protection mechanisms, because the page is marked as supervisor access only. A wild pointer in the kernel? True, that can access any virtual address currently mapped for the process, but the majority of addresses won't be mapped at all, meaning that a page fault will be caught by the memory manager, which presumably can determine that the page shouldn't be accessible and raise a protection fault. If it does hit an address that is live, then yes, a kernel bug can have the effect you describe - but the same is just as true with segmentation. A corrupted supervisor-mode pointer is a supervisor-mode pointer, period.

Yes, most RTOSes will use single-address-space identity mapping (due to the unpredictable timing inherent in virtual memory), but again, most wild or malicious pointers will end up in memory that is held by other user processes, or by the supervisor code, meaning that only a kernel bug can have the effect you describe. This would apply just as much to a segmentation-based kernel as to one using only paging.

As for drivers, well, either they are running in supervisor mode - whether intrinsic to the kernel as with a monolithic kernel, or loaded as modules, as with most hybrid models - or they are in a separate process, as with a microkernel system. For microkernels, the drivers would be covered by the protection mechanisms the same as the user processes are (even if one were to use the intermediate ring 1 or ring 2 levels). For supervisor-mode drivers - whether loadable or not - then it becomes a matter of trust, again regardless of whether segmentation is used or not.

The only way what you are describing could work is if the driver segments are run in supervisor mode, but mapped separately from the kernel to their own code, stack, and data segments. As far as I am aware, this isn't possible - supervisor-mode memory will all have the same memory mapping, meaning that the kernel would have the same segmentation as the drivers. I can't see any way you can have separate segments within the supervisor memory space for the drivers distinct from the kernel itself - nor can I see how this differs from doing the same with paging, if so. As iansjack said, you can just as easily use separate page tables as you can separate segments.

I will again ask you a question you dodged previously: aside from x86, what other modern ISAs which support virtual memory (i.e., not a microcontroller) have you worked with? It is no coincidence that none of them use segmentation, because more or less all of them have had 32-bit or 64-bit memory addressing from the outset, and didn't need a hack to make a larger address space out of overlapping 16-bit memory addresses. Which is all that x86 segmentation was ever about. Yes, 32-bit protected mode went on to provide 32-bit segments, but that was only for backwards compatibility (the new mode had to have a GDT and LDTs which were supersets of those from 16-bit protected mode, otherwise, it couldn't run existing 16-bit protected mode code), not because they anticipated anyone using segmentation in 32-bit protected mode.

What are you going to do if - or rather, when - Intel drops 32-bit protected mode, the same way they apparently plan to drop real mode? True, it probably won't be any time soon, but it is almost certainly coming - assuming that x86 remains the dominant desktop platform in the first place, which is increasingly unlikely with the growth of ARM platforms with comparable performance to the best x86-64 CPUs. What will you do if there are no more segmented platforms in common use?