64-bit UEFI and 32-bit (segmented) OSes

[rdos]

OK, so I've made the first steps in UEFI booting. I can mount the GPT partitions, and especially the EFI partition which contains the boot efi application. The problem on my HP Pavilion Laptop, which I'll use to debug UEFI booting, is that it can only boot a 64-bit EFI image, and it must always be called efi/boot/bootx64.efi. That means I need some compiler that can output 64-bit code (OpenWatcom currently cannot), and I also need to switch back to non-paged 32-bit mode in order to boot my OS.

In order to check how the boot process works, I checked a couple of existing efi-applications, like rEFInd and rEFIt. I really like that boot manager, and maybe it would be a good idea to reuse it in order to be able to boot other OSes.

I think the best way to maintain bootable RDOS images is by placing them in the EFI partition, like in /rdos directory. I also need a 2:nd stage 32-bit image that sets up the environment properly.

I think this is the way it should operate:
- In the boot menu, all bootable RDOS images in /rdos directory are shown.
- When one of the RDOS images are selected, a memory block in the lower 4G memory area is allocated with a size of 64k + size of boot image.
- The 2:nd stage loader is read into part of the 64k buffer and the boot image is loaded at offset 64k in the memory block.
- Get, process and save UEFI memory and GUI information
- ExitBootServices is called
- A new GDT is created in the 64k buffer and loaded
- Control is passed to legacy mode using the new GDT, and cs:eip is set to the 2:nd stage loader (using segmentation so no relocations are necessary)
- Paging is disabled
- The 2:nd stage loader copies the memory block to 0x111000, with the boot image at 0x121000, which is required in order to boot RDOS with 64-bit support
- The GDT is updated and control is passed to the new image
- The necessary system information is gathered for the RDOS boot code
- Control is transfered to the kernel, and now booting is done as usually.

A big problem is that I don't know where the allocated block is located, and if it overlaps the fixed memory I need to use. Maybe I also need to copy the 2:nd stage loader to some fixed address, like 0x80000 or something before doing the move to a fixed memory area?

Comments?

[Brendan]

Hi,

- In the boot menu, all bootable RDOS images in /rdos directory are shown.

There are 2 cases:

• There is only one OS and no boot menu is needed in the first place (e.g. firmware starts OS directly)
• There is multiple OSs and the boot menu can not be considered an integral part of any OS

My advice here is to either don't bother providing a boot menu thing with the OS at all (let people use any they like), or to provide a generic/optional one in case the user doesn't already have one installed (which could be any open source boot menu thing).

More practically, you're probably going to want an open source boot menu thing that is digitally signed with Microsoft's key as part of a solution to the "secure boot" mess (so that firmware can start boot menu thing, and boot menu thing can then use your OSs key to check your OS and start your OS securely).

- When one of the RDOS images are selected, a memory block in the lower 4G memory area is allocated with a size of 64k + size of boot image.
- The 2:nd stage loader is read into part of the 64k buffer and the boot image is loaded at offset 64k in the memory block.
- Get, process and save UEFI memory and GUI information
- ExitBootServices is called
- A new GDT is created in the 64k buffer and loaded
- Control is passed to legacy mode using the new GDT, and cs:eip is set to the 2:nd stage loader (using segmentation so no relocations are necessary)
- Paging is disabled

You're going to need (relocatable) 64-bit code, that switches from the firmware's GDT to your GDT, then switches to your 32-bit code segment (in long mode). Your 32-bit code would disable long mode.

- The 2:nd stage loader copies the memory block to 0x111000, with the boot image at 0x121000, which is required in order to boot RDOS with 64-bit support

Do not assume anything about the physical address space. There are no guarantees that the areas at 0x111000 or 0x121000 are usable RAM.

A big problem is that I don't know where the allocated block is located, and if it overlaps the fixed memory I need to use. Maybe I also need to copy the 2:nd stage loader to some fixed address, like 0x80000 or something before doing the move to a fixed memory area?

I'd only used fixed virtual addresses (and not use fixed physical addresses). This means using either paging or segmentation for the 2nd stage loader. If your existing 2nd stage requires a fixed virtual address, then write a new 2nd stage for UEFI that doesn't.


Cheers,

Brendan

[rdos]

There are 2 cases:

• There is only one OS and no boot menu is needed in the first place (e.g. firmware starts OS directly)
• There is multiple OSs and the boot menu can not be considered an integral part of any OS

My advice here is to either don't bother providing a boot menu thing with the OS at all (let people use any they like), or to provide a generic/optional one in case the user doesn't already have one installed (which could be any open source boot menu thing).

More practically, you're probably going to want an open source boot menu thing that is digitally signed with Microsoft's key as part of a solution to the "secure boot" mess (so that firmware can start boot menu thing, and boot menu thing can then use your OSs key to check your OS and start your OS securely).

I already have a boot-manager in the BIOS boot solution, and I want to keep that. The idea is that you always have two bootable images, one called "standard" and one called "safe". If you download a non-working "standard" image, you can always boot the "safe" version and retry. Sometimes I use more than two images. These images are self-contained and typically a few MB large, so can easily fit into the EFI partition that is typically a few 100MBs.

Because of this requirement, it might be a good idea to integrate with some boot-manager. In GRUB, for instance, I can create the multiple boots by adding two sections in the config-file. I also like the feature to be able to boot any efi + boot from USB stick without a need to go through the BIOS setup screen.

As for secure boot, I won't bother about that, and instead just turn it off (already did + wiped the keys).

You're going to need (relocatable) 64-bit code, that switches from the firmware's GDT to your GDT, then switches to your 32-bit code segment (in long mode). Your 32-bit code would disable long mode.

Yes, but that is only a few assembly instructions for loading GDT, and a far jump to the legacy code selector. I have this code in my 64-bit extension already.

Do not assume anything about the physical address space. There are no guarantees that the areas at 0x111000 or 0x121000 are usable RAM.

I'd only used fixed virtual addresses (and not use fixed physical addresses). This means using either paging or segmentation for the 2nd stage loader. If your existing 2nd stage requires a fixed virtual address, then write a new 2nd stage for UEFI that doesn't.

They must be, otherwise things will not work.

Because the switcher between long mode and protected mode (and reverse) must turn off paging, it must execute at some known physical addresses. I assigned the above range for that.

The 2nd stage loader could run at any position, but since 64-bit support must have a fixed physical range, it should be no additional problem to reuse that in the boot process.

[Combuster]

Because the switcher between long mode and protected mode (and reverse) must turn off paging, it must execute at some known physical addresses.

The only requirement is linear=physical, which is easily obtained as that's part of UEFI in the first place. In fact, mov eax, cr0; and eax, ~cr0_pg; mov cr0, eax; is 100% void of dependencies on specific addresses.

[rdos]

Because the switcher between long mode and protected mode (and reverse) must turn off paging, it must execute at some known physical addresses.

The only requirement is linear=physical, which is easily obtained as that's part of UEFI in the first place. In fact, mov eax, cr0; and eax, ~cr0_pg; mov cr0, eax; is 100% void of dependencies on specific addresses.

In UEFI, yes, but not when the OS is up and running with a non-identity page mapping. The reservation of a fixed physical address range exists so that the OS can dynamically turn off paging and change operating mode as a core is scheduled to another process with a different bitness.

Also, the above is only valid if the UEFI code executes below 4G, which I'm not sure is guaranteed if a 64-bit UEFI is running.

[Brendan]

Hi,

As for secure boot, I won't bother about that, and instead just turn it off (already did + wiped the keys).

If my OS was installed in a dual-boot system next to something like Windows, I'd want secure boot enabled to protect my OS from all the software abusing security holes in Windows...

Do not assume anything about the physical address space. There are no guarantees that the areas at 0x111000 or 0x121000 are usable RAM.

I'd only used fixed virtual addresses (and not use fixed physical addresses). This means using either paging or segmentation for the 2nd stage loader. If your existing 2nd stage requires a fixed virtualphysical address, then write a new 2nd stage for UEFI that doesn't.

They must be, otherwise things will not work.

Because the switcher between long mode and protected mode (and reverse) must turn off paging, it must execute at some known physical addresses. I assigned the above range for that.

The 2nd stage loader could run at any position, but since 64-bit support must have a fixed physical range, it should be no additional problem to reuse that in the boot process.

I think you need to talk to someone that knows how to use segmentation.

64-bit relocatable code can easily adjust a 32-bit segment's base address, then "jump far" to that 32-bit segment; such that the 32-bit code ends up using fixed offsets from the segment's base and not fixed physical addresses.


Cheers,

Brendan

[rdos]

I think another method for loading the 2nd stage loader could be better:
1. Allocate a separate buffer for the 2nd stage loader
2. Move the buffers only if it contains the reserved physical area
3. Put the GDT in the 0x111000 area as before, as this would reuse the fixed memoy area.

Maybe with this method, I won't need to move the boot image itself, and it will work at any position (except if it covers the 0x111000 to 0x120fff area). So, I'll do a move on the boot image and 2nd stage loader only if it covers the fixed physical area.

[rdos]

They must be, otherwise things will not work.

Because the switcher between long mode and protected mode (and reverse) must turn off paging, it must execute at some known physical addresses. I assigned the above range for that.

The 2nd stage loader could run at any position, but since 64-bit support must have a fixed physical range, it should be no additional problem to reuse that in the boot process.

I think you need to talk to someone that knows how to use segmentation.

64-bit relocatable code can easily adjust a 32-bit segment's base address, then "jump far" to that 32-bit segment; such that the 32-bit code ends up using fixed offsets from the segment's base and not fixed physical addresses.

Would work except for the fact that switching between protected mode and long mode (and back) can only by done by disabling paging, so this code must be in a unity-mapped memory area. And since the area must be unity-mapped, you need to know both the linear and physical address where this code resides, and the easiest way to do this is to reserve a range of physical memory, and the same linear memory range. In UEFI, this is not an issue since all memory is unity-mapped.

[Brendan]

Hi,

They must be, otherwise things will not work.

Because the switcher between long mode and protected mode (and reverse) must turn off paging, it must execute at some known physical addresses. I assigned the above range for that.

The 2nd stage loader could run at any position, but since 64-bit support must have a fixed physical range, it should be no additional problem to reuse that in the boot process.

I think you need to talk to someone that knows how to use segmentation.

64-bit relocatable code can easily adjust a 32-bit segment's base address, then "jump far" to that 32-bit segment; such that the 32-bit code ends up using fixed offsets from the segment's base and not fixed physical addresses.

Would work except for the fact that switching between protected mode and long mode (and back) can only by done by disabling paging, so this code must be in a unity-mapped memory area. And since the area must be unity-mapped, you need to know both the linear and physical address where this code resides, and the easiest way to do this is to reserve a range of physical memory, and the same linear memory range. In UEFI, this is not an issue since all memory is unity-mapped.

It would work. The fact that you need to be in identity mapped memory when your 32-bit code switches off long mode and paging doesn't change the fact that you don't need to care what the physical (and virtual) address of the code is and don't need a fixed physical address.

Basically:

• You're in long mode running position independent 64-bit code, with identity mapped paging
• You configure a 32-bit descriptor such that "segment base = virtual address of code = physical address of code"
• You switch to 32-bit code running at the same virtual address ("segment base + offset = same virtual address as before"), with identity mapped paging
• You disable long mode and paging, which makes almost no difference because it was identity mapped to begin with ("segment base + offset = physical address = same as the virtual address before paging was disabled")

Cheers,

Bredan

[jnc100]

That means I need some compiler that can output 64-bit code (OpenWatcom currently cannot)

May I suggest a gcc that targets 'x86_64-w64-mingw32' which is a reasonable readily available alternative (on most systems), in conjunction with the headers from gnu-efi (but don't use the gnu-efi build system itself).

To use:

Code: Select all

x86_64-w64-mingw32-gcc -ffreestanding -mabi=ms <usual plethora of options to disable mmx, sse etc and enable warnings> -c -o file.o file.c
x86_64-w64-mingw32-gcc -nostdlib -Wl,-dll -shared -Wl,--subsystem,10 -e efi_main -o bootx64.efi file.o

Regards,
John.

[Gigasoft]

If you have to use a fixed address, at least use one below 80000h, so that you know it's always a valid address on a PC. In my OS, this code resides at a variable address, depending on where the boot sector decided to load the OS. The first 1MB in the system address space is identity mapped and the physical pages corresponding to the part of the OS that remains in low memory are marked as reserved. No linear reservation is necessary, as the first 3GB of the system address space are unused (except for the few things that need to be identity mapped).

[rdos]

If you have to use a fixed address, at least use one below 80000h, so that you know it's always a valid address on a PC. In my OS, this code resides at a variable address, depending on where the boot sector decided to load the OS. The first 1MB in the system address space is identity mapped and the physical pages corresponding to the part of the OS that remains in low memory are marked as reserved. No linear reservation is necessary, as the first 3GB of the system address space are unused (except for the few things that need to be identity mapped).

This fixed address is only required if long mode support is loaded, and thus if the processor supports 64-bit operation. I don't think that on such modern machines the 111000h to 120fff range is ever used for anything else.

Besides, it seems like there is a quick EFI fix for this. You can actually reserve an address range, and if this fails RDOS could simply refuse to boot. I don't know of any modern PCs were this address range is not free.

[jnc100]

Besides, it seems like there is a quick EFI fix for this. You can actually reserve an address range, and if this fails RDOS could simply refuse to boot. I don't know of any modern PCs were this address range is not free.

The problem with that is that any of the multitude of loaded EFI drivers may have already reserved that memory range for itself. It may not have even asked for it especially, but just been randomly allocated it when it requested any chunk of memory of a certain length.

As has been highlighted above I would try to make your paging-disabling code position-independent, which is certainly not difficult.

Regards,
John.

[rdos]

Besides, it seems like there is a quick EFI fix for this. You can actually reserve an address range, and if this fails RDOS could simply refuse to boot. I don't know of any modern PCs were this address range is not free.

The problem with that is that any of the multitude of loaded EFI drivers may have already reserved that memory range for itself. It may not have even asked for it especially, but just been randomly allocated it when it requested any chunk of memory of a certain length.

As has been highlighted above I would try to make your paging-disabling code position-independent, which is certainly not difficult.

Yes, I see the problem. Another solution is to try to reserve the physical page-disabling range, and then allocating a buffer for the 2nd stage loader and OS image. This would guarantee that the OS image would not overlap the physical page-disabling range, and if some EFI drivers are loaded there doesn't matter (they can be wiped-out after calling ExitBootservices). I have no problem loading the OS image at whatever position is free (segmentation solves this so relocations aren't necessary).

[Brendan]

Hi,

Besides, it seems like there is a quick EFI fix for this. You can actually reserve an address range, and if this fails RDOS could simply refuse to boot. I don't know of any modern PCs were this address range is not free.

The problem with that is that any of the multitude of loaded EFI drivers may have already reserved that memory range for itself. It may not have even asked for it especially, but just been randomly allocated it when it requested any chunk of memory of a certain length.

As has been highlighted above I would try to make your paging-disabling code position-independent, which is certainly not difficult.

Yes, I see the problem.

I don't think you do, at least not completely.

If you forget about obsolete assumptions about what the physical address space used to look like for BIOS, the only thing you can really look at to determine what might happen in future UEFI systems is the UEFI spec itself. For example; if the entire first 1 GiB of the physical address space was reserved for PCI devices and doesn't contain any RAM whatsoever, then that doesn't violate anything in the UEFI spec, and therefore it's entirely possible that a UEFI system might do that in future.

Of course that's a relatively unlikely scenario. Less unlikely would be UEFI firmware that uses the area for ACPI NVS or SMRAM or something else.

Basically, you can't predict the future; so for well designed software the goal is to minimise risk.

To minimise risk, don't use a fixed physical addresses.


Cheers,

Brendan