Protecting the OS

[d2alphame]

I'm writing a 64 bits operating system. The memory model is flat with paging. I'd like to know what techniques and methods members of the community use to protect System-Level Applications from each other, since these applications will be running at privilege level 0. I would like my OS to run applications at level 0, and at the same time protect the OS from those applications.

[dozniak]

I would like my OS to run applications at level 0, and at the same time protect the OS from those applications.

You can't.

[Love4Boobies]

Of course you can. You just need to isolate the processes in software in the same way managed language implementations protect against illegal memory accesses.

[Brendan]

Hi,

I'm writing a 64 bits operating system. The memory model is flat with paging. I'd like to know what techniques and methods members of the community use to protect System-Level Applications from each other, since these applications will be running at privilege level 0. I would like my OS to run applications at level 0, and at the same time protect the OS from those applications.

Any software that's running at CPL=0 has access to everything; and any application running at CPL=0 has access to everything the kernel itself has access to. There are only about 4 ways to try to prevent these applications from doing anything they like (including trashing the kernel, messing with MSRs, reconfiguring the chipset, disabling long mode/paging, etc):

• Use a special language and toolchain that doesn't allow unsafe code to be created (e.g. managed code)
• run the kernel as a hyper-visor (using hardware virtualisation to protect the host from the guest/s)
• make sure all code running at CPL=0 has no bugs and is open source, and have protected/secure software distribution (to prevent "man in the middle" malicious code)
• make sure all code running at CPL=0 has no bugs and have some system to ensure only verified software can be run (e.g. require digital certificates)

None of these options are perfect. Managed code takes a huge amount of work and remains vulnerable to compiler bugs. Running the kernel as a hyper-visor is silly - a more traditional micro-kernel with "system applications" (drivers, etc) running at CPL=3 is a lot easier (and doesn't require CPUs that support hardware virtualisation). The last 2 options aren't really sane (there's no easy way to guarantee there's no bugs); the open source model assumes that people actually look at the source code and understand it (people can't/don't do this, so "many eyes" doesn't do much more than provide a false sense of security), and the "digital certificates" method means that the person who issues the digital certificates needs to be able to guarantee the software is correct (e.g. not malicious, no deliberate back-doors, etc) which is expensive (how much are you paying someone to verify software?) and prone to false positives (it's impossible to guarantee that no problems were missed so you risk ending up with certified malicious code).

The alternative is to minimise the amount of code that runs at CPL=0, and therefore minimise the amount of code that needs to be trusted. This is the basic idea of micro-kernels; where only a small kernel runs at CPL=0 and everything else (device drivers, etc) run at CPL=3. In general this costs a little performance (due to extra overhead in the communication between separate pieces of software), and (unless you're able to use IOMMUs to prevent drivers that use DMA or bus mastering from bypassing security) may not prevent 100% of all possible problems; but the overhead can be very low and there can be other benefits (flexibility, fault tolerance, scalability).


Cheers,

Brendan

[d2alphame]

Thanks @Brendan siiiigghh... I should have been the one to design the x64 system!

[Love4Boobies]

Actually, you can formally verify your compiler implementation. For instance, check out CompCert and the related Verified Software Toolchain.

[rdos]

If you want protection between code running at CPL=0 you need to use 32-bit protected mode with segmentation (or legacy mode within long mode). Long mode has no such feature.

[Antti]

If you want protection between code running at CPL=0 you need to use 32-bit protected mode with segmentation

It does not protect the system from malicious code. However, it can protect the system from bugs.

[madanra]

I would like my OS to run applications at level 0, and at the same time protect the OS from those applications.

"Running at level 0" means "has full control of the computer" - so what you're asking is for applications to have full control of the computer, but be protected from each other, which is a contradiction in terms.

[Love4Boobies]

No, it isn't. Read the thread.

[madanra]

No, it isn't. Read the thread.

Which bit isn't? The "has full control of the computer", or "is a contradiction in terms", or something in my understanding of the question?
(Though thinking again, I do realise my answer was pointless, as Brendan's answer is a much more nuanced version of what I was intending to mean.)

[d2alphame]

Ok, so I'm thinking actually apps running at CPL 0 CANNOT actually be protected from each other. But I could provide some level of protection for the OS by placing it's code and data in a page that is executable + read but cannot be written to (write-protected). I think this at least puts a small sort of caution in place for unintentional interference with the OS by apps running at CPL 0.

[dozniak]

some level of protection for the OS by placing it's code and data in a page that is executable + read but cannot be written to (write-protected). I think this at least puts a small sort of caution in place for unintentional interference with the OS by apps running at CPL 0.

You do realise that this protection attribute can be changed at any time by any other code also running in ring0?

It could protect from unintended modification, but that's only small part of the whole protection.

[d2alphame]

You do realise that this protection attribute can be changed at any time by any other code also running in ring0?

Yes of course. And that's why I first pointed out that ...apps running at CPL 0 CANNOT actually be protected from each other

It could protect from unintended modification, but that's only small part of the whole protection.

Yes I realize that. But what can I do?

[rdos]

If you want protection between code running at CPL=0 you need to use 32-bit protected mode with segmentation

It does not protect the system from malicious code. However, it can protect the system from bugs.

Combine it with requiring all CPL=0 code being contained in a signed binary, and that can be solved as well. Of course, that precludes loading drivers dynamically from disc, but if you want security you cannot allow such things anyway.