64-bit UEFI and 32-bit (segmented) OSes

[rdos]

I just discovered something interesting about UEFI. It seems like my HP laptop can be partitioned with GPT, and boot with UEFI, but still have the VBE interface. The trick in achieving this seems to be to format the disc with GPT, create an EFI system partition, and then turn off UEFI boot. With this setup I'm able to install Windows 7 on GPT, and also CentOS (Linux) with UEFI. Actually, I started the installation in UEFI mode with CentOS, and then moved to legacy-mode and installed Windows 7. With these settings I can boot RDOS from USB disc (MBR partitioned), Windows 7 through EFI loader without GOP and CentOS with EFI loader and GOP. This seems to be an optimal environment for experimenting with UEFI and doing a gradual migration.

[turdus]

I've started the UEFI road, and have got quite far. If I may I'd give some advice.

First, there's no need for 2nd stage on UEFI, one efi boot loader application is more than enough.

According to /efi/boot/bootx64.efi: you should write your boot loader application in a way knowing it will be run under different names (shouldn't be a problem), and by default, put it in /rdos/bootx64.efi. If you don't want to use a boot manager, simply copy it in /efi/boot directory, but if you do, you can make that to load your app in /rdos. Either way when your app gains control it can load your OS, and don't care whether a boot manager is installed at all.

About virtual-linear-physical memory problem: the easiest thing to use the identity mapping offered by UEFI and call UEFI's allocate function to get a memory to work with. This way no other UEFI driver will interfere, and because of identity mapping you will have the physical memory address as well. If your code needs to access memory with paging turned off, you'll need to do a little bit of assembly and use registers with calculated offsets to do so, as you cannot rely on inline constant offsets generated by your linker. (In other words with this trick you can forget Brendan's warning "I'd only used fixed virtual addresses (and not use fixed physical addresses".)

But, also consider:
1. there's no guarantee you can get any free memory under 4G (for non-paged protected mode), no trick help with that
2. all BIOS functions have UEFI equivalents
3. no way to know how long the BIOS legacy code (such as VBE) will work on modern machines
therefore there's no need, and you should never switch CPU mode on UEFI architecture in the first place.

As for the OS image: you can load it from any FAT device using UEFI calls (no need to implement fs driver in your loader), and let UEFI allocate the memory for it. Once you turn off identity mapping, and set something more comfortable arrangement for your kernel, you can map it at whatever address you like. Just make sure you call exitbootservices before you do so to stop other UEFI applications. If you plan to use fs other than FAT (like me), use the UEFI allocate and readblocks calls and you're good to go. No need for any non-paged mode code to load your kernel at a fixed address.

Think it that way: forget code reuse, you'll need to completely rewrite your booting code in a single UEFI application that provides exactly the same environment as your current BIOS loader. (By using PMBR as stage1 you'll still be able to boot the same USB disk image on BIOS machines as well as on UEFI ones without your kernel even knowing which one was used to boot it).

[rdos]

I've started the UEFI road, and have got quite far. If I may I'd give some advice.

First, there's no need for 2nd stage on UEFI, one efi boot loader application is more than enough.

The 2nd stage needs to run in 32-bit protected mode, preferentially with a segmented model to get rid of problems of where the code is loaded. The kernel is started in non-paged 32-bit protected mode by creating selector 0x30, mapping it to the kernel code loaded in memory, and doing a call to the entry-point. There is no other way to load RDOS.

According to /efi/boot/bootx64.efi: you should write your boot loader application in a way knowing it will be run under different names (shouldn't be a problem), and by default, put it in /rdos/bootx64.efi. If you don't want to use a boot manager, simply copy it in /efi/boot directory, but if you do, you can make that to load your app in /rdos. Either way when your app gains control it can load your OS, and don't care whether a boot manager is installed at all.

Yes, except there seems to be some magic also. It seems like Windows 7 can hijack this process and force UEFI to always run some specific efi-file.

But, also consider:
1. there's no guarantee you can get any free memory under 4G (for non-paged protected mode), no trick help with that

There is. You refuse to boot

The memory allocation function can be passed a suggested address, and UEFI should reserve it if possible. It can also be passed an upper limit (for instance 4G). This can be used both for the fixed physical address, and to obtain memory in the correct range.

2. all BIOS functions have UEFI equivalents
3. no way to know how long the BIOS legacy code (such as VBE) will work on modern machines
therefore there's no need, and you should never switch CPU mode on UEFI architecture in the first place.

Right, but I need a convenient migration route. It's much easier to debug an UEFI implementation that also doesn't need to assume VBE is not present. It's preferable to be able to first boot with UEFI and VBE, implement the required text-mode functions with LFB, and then boot with UEFI and GOP.

As for the OS image: you can load it from any FAT device using UEFI calls (no need to implement fs driver in your loader), and let UEFI allocate the memory for it. Once you turn off identity mapping, and set something more comfortable arrangement for your kernel, you can map it at whatever address you like. Just make sure you call exitbootservices before you do so to stop other UEFI applications. If you plan to use fs other than FAT (like me), use the UEFI allocate and readblocks calls and you're good to go. No need for any non-paged mode code to load your kernel at a fixed address.

There is a need to load it below 4G. Otherwise it will be impossible to start the OS.

Think it that way: forget code reuse, you'll need to completely rewrite your booting code in a single UEFI application that provides exactly the same environment as your current BIOS loader. (By using PMBR as stage1 you'll still be able to boot the same USB disk image on BIOS machines as well as on UEFI ones without your kernel even knowing which one was used to boot it).

There is no way I will do a full 64-bit OS at the same time as an UEFI boot. If the UEFI boot code cannot handle the current 32-bit segmented kernel, it's simply useless.

[rdos]

Updated logic for allocating memory during boot:
1. Reserve all memory up to 0x121000 (using a page-at-a-time allocation request with suggested address)
2. Allocate memory for the boot image, giving UEFI an upper limit of 4G
3. Read boot image to address returned by #2
4. Read-out physical address map and save
5. Allocate memory for temporary 2nd stage
6. Read 2nd stage loader to address returned by #5
7. Call ExitBootServices
8. Copy 2nd stage loader to 0x111000
9. Transfer control to 2nd stage loader

I think this should be safe. Possibly 8 can be omitted, and control be could be passed to protected mode at an arbitrary address directly in the area allocated by #5.

[rdos]

If you forget about obsolete assumptions about what the physical address space used to look like for BIOS, the only thing you can really look at to determine what might happen in future UEFI systems is the UEFI spec itself. For example; if the entire first 1 GiB of the physical address space was reserved for PCI devices and doesn't contain any RAM whatsoever, then that doesn't violate anything in the UEFI spec, and therefore it's entirely possible that a UEFI system might do that in future.

I think I have an implicit assumption in the kernel initialization code that at least a few pages at the bottom of memory are RAM. I can work around that, but for now this seems like a minor problem.

Of course that's a relatively unlikely scenario. Less unlikely would be UEFI firmware that uses the area for ACPI NVS or SMRAM or something else.

UEFI mapping ACPI above 4G could be a problem. Likewise if they map it somewhere below 0x121000 in a non-standard area. OTOH, the latter seems improbable.

[Brendan]

Hi,

Updated logic for allocating memory during boot:
1. Reserve all memory up to 0x121000 (using a page-at-a-time allocation request with suggested address)
2. Allocate memory for the boot image, giving UEFI an upper limit of 4G
3. Read boot image to address returned by #2
4. Read-out physical address map and save
5. Allocate memory for temporary 2nd stage

Allocating memory will change the physical address map (possibly including UEFI moving things around to make space), so whatever you saved in step 4 is stale/wrong after step 5.

6. Read 2nd stage loader to address returned by #5
7. Call ExitBootServices
8. Copy 2nd stage loader to 0x111000
9. Transfer control to 2nd stage loader[/quote]

Seems silly to load something in the wrong place and then copy it.

Updated updated logic:
1. Determine size of 2nd stage
2. Allocate memory at 0x111000 for 2nd stage; including gracefully aborting boot (with a nice descriptive apology for the user) if this fails
3. Load 2nd stage at 0x111000
4. Allocate a small amount of memory (1 page?) anywhere below 4 GiB and copy "trampoline code" into it, so that it can be used to safely switch to 32-bit
5. Read-out physical address map and save
6. Call ExitBootServices
7. Use "trampoline code" to switch to 32-bit safely
8. Transfer control to 2nd stage loader[/quote]

If you forget about obsolete assumptions about what the physical address space used to look like for BIOS, the only thing you can really look at to determine what might happen in future UEFI systems is the UEFI spec itself. For example; if the entire first 1 GiB of the physical address space was reserved for PCI devices and doesn't contain any RAM whatsoever, then that doesn't violate anything in the UEFI spec, and therefore it's entirely possible that a UEFI system might do that in future.

I think I have an implicit assumption in the kernel initialization code that at least a few pages at the bottom of memory are RAM. I can work around that, but for now this seems like a minor problem.

Yes, a minor problem compared to assuming the area at 0x111000 is usable RAM (and not faulty RAM, ROM, a memory mapped device, code/data needed by UEFI run-time services, ACPI NVS, SMRAM, etc).

Of course that's a relatively unlikely scenario. Less unlikely would be UEFI firmware that uses the area for ACPI NVS or SMRAM or something else.

UEFI mapping ACPI above 4G could be a problem. Likewise if they map it somewhere below 0x121000 in a non-standard area. OTOH, the latter seems improbable.

Where in the UEFI specification does it say that mapping ACPI below 0x121000 is non-standard? The UEFI specification doesn't give any recommendation for the location/s of anything, and no address is more or less standard than any other.


Cheers,

Brendan

[rdos]

Updated updated logic:
1. Determine size of 2nd stage
2. Allocate memory at 0x111000 for 2nd stage; including gracefully aborting boot (with a nice descriptive apology for the user) if this fails
3. Load 2nd stage at 0x111000
4. Allocate a small amount of memory (1 page?) anywhere below 4 GiB and copy "trampoline code" into it, so that it can be used to safely switch to 32-bit
5. Read-out physical address map and save
6. Call ExitBootServices
7. Use "trampoline code" to switch to 32-bit safely
8. Transfer control to 2nd stage loader

Yes. Maybe it is a better idea to assume 0x111000 can be used, and aborting if not. However, 0x111000 IS the trampoline code, so #4 is not necessary.

Where in the UEFI specification does it say that mapping ACPI below 0x121000 is non-standard? The UEFI specification doesn't give any recommendation for the location/s of anything, and no address is more or less standard than any other.

Right. The safe way to go about ACPI would be to relocate it to a safe place while still in UEFI. Not the first priority, but still necessary.

[Owen]

Right. The safe way to go about ACPI would be to relocate it to a safe place while still in UEFI. Not the first priority, but still necessary.

You can't relocate things like ACPI NVS.

Yes, except there seems to be some magic also. It seems like Windows 7 can hijack this process and force UEFI to always run some specific efi-file.

Consult the UEFI speicifcation. Specifically the specification of the BOOT#### variables.

Be aware that some machines ignore these completely (noncompliant, but it works with Windows because Windows installs /efi/boot/bootx64.efi). To work around the deficiencies of these machines, consider installing some form of general UEFI boot manager as /efi/boot/bootx64.efi (i.e. one which does the job the incompetent firmware vendor failed to do).