32-bit kernel that runs 64-bit apps

[Antti]

has turned into a very silly discussion

Why this discussion is silly? This is a highly on-topic discussion.

We could have a kernel written in bytecode and have the "little" glue that interprets it and take care of all the hardware specific things. If that bytecode is "8-bit", would that kernel be "8-bit kernel" that runs 32-bit, 64-bit and 128-bit applications? Or would the platform that allows running the bytecode, e.g. the 64-bit "little" glue, be the real background kernel?

[iansjack]

Why this discussion is silly?

Silly because, as Gigasoft points out, it ignores the easily checked facts. There are a host of theoretical explanations as to why it cannot possibly work, ignoring the simple fact that it did. There are demands for citations to prove it works, but a glaring lack of citations to support the opposite opinion.

It is, as I have said before, the equivalent of scientists proving that bumble bees can't fly. And yet they do. Eppur si muove.

Many operating systems contain a mixture of 16, 32, and 64-bit code. That doesn't make them 16-bit, 32-bit, or - necessarily - 64-bit. I guess if we are to define the "bitness" of an operating system then we have to determine the minimum processor requirements. Can the kernel run on an x-bit processor? In the case of OS X we had.

1. A kernel that would run on 32-bit or 64-bit processors.
2. A kernel that would only run on 64-bit processors.

To my mind the only sensible way to categorize those kernels is to do as Apple did and call kernel 1 a 32-bit kernel and kernel 2 a 64-bit kernel. Kernel 1 ran 32-bit device drivers, kernel 2 required 64-bit device drivers (which didn't exist in many cases). Kernel 1 had the ability when running on a 64-bit processor to run 64-bit user programs. But the kernel itself was still, to all intents and purposes, 32-bit. You could take a hard disk with that kernel installed from a 64-bit machine, where it would run 64-bit user programs, to a 32-bit machine, where it wouldn't. But that didn't change the kernel - it only required a 32-bit machine to run so it was a 32-bit kernel.

It is, if you like, just a matter of semantics. And just a bit of irrelevant history. The only real reason for such an implementation was the existence of a huge history of 32-bit device drivers coupled with the desire to run more efficient 64-bit user programs. I can think of no reason to design an OS like that from scratch, merely as a stepping stone to a full-blown 64-bit implementation.

[Antti]

I guess if we are to define the "bitness" of an operating system then we have to determine the minimum processor requirements. Can the kernel run on an x-bit processor? In the case of OS X we had.

1. A kernel that would run on 32-bit or 64-bit processors.
2. A kernel that would only run on 64-bit processors.

Does not make sense. Look at this pseudocode:

Code: Select all

KERNEL 1:

	if (CheckCPU() == "32-bit") {
		Start32BitKernelVersion();
	} else if (CheckCPU() == "64-bit") {
		Start64BitKernelVersion();
	}


KERNEL 2:

	if (CheckCPU() == "32-bit") {
		/* NOT SUPPORTED */
	} else if (CheckCPU() == "64-bit") {
		Start64BitKernelVersion();
	}

When running Kernel 1 on a 64-bit CPU, is this running instance "a 32-bit" kernel?

[iansjack]

Your pseudocode is, I'm afraid, flawed. It wasn't a case of loading a different kernel. Both 32-bit and 64-bit processors loaded exactly the same kernel if you chose the 32-bit version (which was the default). They didn't load different code. Fairly obviously, that kernel code was 32-bit code. There was a separate 64-bit kernel that you could load, if you chose to, on 64-bit processors.

A kernel that can run on 32- or 64-bit processors is a very common concept - the world is full of them. Windows, Linux, FreeBSD, for example, all have 32-bit versions. Those versions will run on 64-bit processors. But, unlike the 32-bit OS X kernel they cannot run 64-bit programs.

The silliness continues. You are starting with the stone cold certainty that a 32-bit kernel can't run 64-bit programs and then setting out to prove that certainty with flawed assumptions. I'm never going to convince you but I don't really care and I'm not going to try further.

[Antti]

It was not about loading but branching the execution path. The point of my Kernel 1 pseudocode was that if the CPU supported 64-bit mode, it would behave differently. This means that the "32-bit" kernel is really a 32 bit kernel or a 64-bit kernel, depending on the CPU.

The silliness continues.

I do not understand this "silliness" thing you have mentioned several times. You have said you do not like "beginner" questions on this forum and all that. Now we have a "real" discussion and you are trying to stop it immediately.

[Candy]

If you take the definition as "32-bit only kernel" then it is impossible, as shown above. If you take it as "32-bit kernel that is 64-bit able", then it is trivially possible, in the same way that Win95 could run 16-bit apps.

You're arguing over semantics. On 30+ posts.

[iansjack]

Nothing of the sort has been shown above. The only real fact that we have to go on is that Apple's 32-bit OS X kernel could run 64-bit user programs. The rest is conjecture. Unlike the facts about OS X it is not backed by any citations.

Don't mistake that which you are unfamiliar with with the impossible. It goes without saying that for a kernel to run 64-bit programs it must have some awareness of how those programs run; that doesn't mean that it has to run in the same way. To take a simple example, I can use 32-bit code to create a 64-bit page table. The 32-bit code has to be aware of how that table works, but it doesn't use that table itself.

[Brendan]

Hi,

Nothing of the sort has been shown above. The only real fact that we have to go on is that Apple's 32-bit OS X kernel could run 64-bit user programs. The rest is conjecture. Unlike the facts about OS X it is not backed by any citations.

"Apple's 32-bit OS X kernel" is:

• a) not a kernel. It's a task/process running under a modified Mach micro-kernel
  b) doesn't run 64-bit tasks (it asks the modified Mach micro-kernel to run them)
  c) isn't entirely Apple's (at least some code is from FreeBSD)

Not one piece of "Apple's 32-bit OS X kernel" is factually correct.

Don't mistake that which you are unfamiliar with with the impossible.

Today, I saw plans for an electric circuit that needs a little electricity to start, but once started will keep a light globe running forever. I can sell you copy of the plans for a very small fee of $2000 - it's cheap when you consider all the free electricity you'll be getting when you build a huge one!


Cheers,

Brendan

[Owen]

Hi,

Nothing of the sort has been shown above. The only real fact that we have to go on is that Apple's 32-bit OS X kernel could run 64-bit user programs. The rest is conjecture. Unlike the facts about OS X it is not backed by any citations.

"Apple's 32-bit OS X kernel" is:

• a) not a kernel. It's a task/process running under a modified Mach micro-kernel

By the time Darwin was Darwin (and not NeXTSTEP/OPENSTEP) the Mach and BSD layers were already fused together so tightly at the hip that separating them in such a fashion would be impossible. The "BSD layer" isn't a process running under Mach (as it once was); the two are linked together intimately in the "They call into each other and share data structures" sense of that statement. Yes, the "BSD task" remains a semi-logical process, but it doesn't access mach by system calls (and mach doesn't access it by IPC...). Observe that the sys call dispatch layer is fused, for example.

• b) doesn't run 64-bit tasks (it asks the modified Mach micro-kernel to run them)

Considering the above, said statement makes little sense

• c) isn't entirely Apple's (at least some code is from FreeBSD)

Not one piece of "Apple's 32-bit OS X kernel" is factually correct.

I have no idea how this argument is supposed to have any bearing on the question at hand.

One of the things to remember is because of the separate BSD task pedigree, the Darwin kernel (at least on 32-bit systems; I don't think this is the case on 64-bit systems with their now copious address space) runs in a separate address space from the application, with a small layer of code mapped into both to manage the transitions between them (This design was probably in part because, on systems without the horrible MMU behaviour of x86, this has negligible cost). This "Common code" handles most of the complexity of task switches, and needs to know about long mode quite intimately. The rest of the code which needs to deal with such architecture differences is quite well factored by necessity (Remember that at one point they were simultaneously shipping kernels for five architectures from the same code base - ppc, ppc64, i386, amd64 for OS X, armv6 for iPhone OS as it was known at the time) and the VM layer was taught to deal with 64-bit and 32-bit processes (which, realistically, isn't much harder than dealing with legacy paging vs PAE, for example)

[Gigasoft]

"Apple's 32-bit OS X kernel" is:
a) not a kernel. It's a task/process running under a modified Mach micro-kernel
b) doesn't run 64-bit tasks (it asks the modified Mach micro-kernel to run them)
c) isn't entirely Apple's (at least some code is from FreeBSD)
Not one piece of "Apple's 32-bit OS X kernel" is factually correct.

I think it should have been obvious all along that we are talking about the file named mach_kernel. Arguing about how much of it that was really written by Apple and what parts of it you would consider to be the "kernel" does nothing to change the basic facts about how the system works. This file named mach_kernel comes in a 32-bit version and a 64-bit version, and right now we are talking about the 32-bit version. This kernel implements Mach system calls as well as Unix system calls. Neither can be said to be running "under" the other, it's all just one big program which happens to contain code that was originally written by various different companies. When this kernel runs on a 32-bit CPU, it uses 386 or PAE paging, and when it runs on a 64-bit CPU, it uses IA-32e mode. The only parts of the kernel that ever care about this at all are the code in the beginning that sets up the IDT, GDT and TSS, the code that manages page tables, and the exception/interrupt handling code, and the code that saves/restores FPU state. There are also two different versions of the system call handler, to deal with the different calling conventions for 32-bit and 64-bit applications.

[Brendan]

Hi,

Not one piece of "Apple's 32-bit OS X kernel" is factually correct.

I have no idea how this argument is supposed to have any bearing on the question at hand.

The question at hand is whether a 32-bit kernel can run 64-bit applications. The question is not whether a kernel consisting of a mixture of 64-bit code (e.g. that's mapped into all address spaces) and 32-bit code (e.g. that's in its own private address space, like a normal task/process) can run 64-bit applications.

A 32-bit kernel can not run 64-bit applications (unless virtualisation is used in some form) - it is impossible for a variety of reasons, mostly due to the way AMD designed long mode.

A "64-bit and 32-bit" kernel can run 64-bit applications (even if the 64-bit code is relatively small, and even if the 64-bit code isn't used on 32-bit CPUs) but that has nothing to do with the original question.

I do not understand how this is not obvious.


Cheers,

Brendan

[rdos]

Of course a functional 32-bit kernel can be made to run 64-bit applications. On X86, the 32-bit kernel must handle PAE paging in order for it to be possible to map any buffer from 64-bit mode to 32-bit kernel device drivers.

The process is quite simple:
1. Do a 64-bit "driver" that setup 64-bit interrupt handlers (and redirects them to 32-bit)
2. Put the processor in long mode for 64-bit processes and in compability-mode for 32-bit processes
3. Provide sysall redirects from 64-bit to 32-bit.
4. Write new fault handlers for 64-bit. Most of the work here is with the pagefault handler.

As a bonus, you could also implement new 64-bit drivers as needed that are loaded on 64-bit capable hardware. These could use a micro-kernel model. With a micro-kernel model, they can also be used for 32-bit processes.

There is nothing magical about this. It's pretty straight-forward, and doesn't require more than a tiny bit of the time to implement a new 64-bit kernel from scratch (at least if your current kernel is mostly in assembler).

[Candy]

Of course a functional 32-bit kernel can be made to run 64-bit applications. ... The process is quite simple:

In short, to make a 32-bit kernel run 64-bit processes, just add 64-bit process support.

Still confused about what the discussion is actually about. You need to add 64-bit process support to be able to run 64-bit processes; AMD didn't design it backwards-compatible enough to run 64-bit software under 32-bit OSes without modification.

[thepowersgang]

This is just arguing semantics. The "truth" depends on your definition the bitness of the kernel. In my opinion, if the kernel can manage 64-bit apps (which requires 64-bit code to do), it's a 64-bit kernel (even if it does every other operation in 32-bit code).

On a side note - Would your ideas be switching between protected and long mode? Or running 32-bit kernel code in compatibility mode?

[linguofreak]

This is just arguing semantics. The "truth" depends on your definition the bitness of the kernel. In my opinion, if the kernel can manage 64-bit apps (which requires 64-bit code to do), it's a 64-bit kernel (even if it does every other operation in 32-bit code).

I'd define bitness by the bitness of the ABI provided to kernel modules (which will likely be the bitness of most of the code in the kernel): If kernel modules have to be 32-bit, it's a 32 bit kernel, even if there's a bit of 64-bit code to thunk to a 64-bit userland. The only exception I'd make to this is if the kernel requires a 64-bit CPU (rather than just being able to make use of one if present).