Announcing POSIX-UEFI

[bzt]

Hi All,

I don't know about you, but I personally hate GUIDs and find UEFI API ugly as hell. So I've wrapped all that up in a nice and friendly POSIX layer. It is called
POSIX-UEFI.

It's is extremely lightweight (about 12 files, 150K source, and 32K compiled), works on POSIX systems like Linux and provides POSIX interface for your UEFI application. It also sets up the build environment for you, by detecting your compiler (works with both GNU gcc and LLVM CLang), if it need cross-compiler or can use the host's, what flags to use, etc.

A traditional Hello World example with this library:

Code: Select all

#include <uefi.h>
 
int main (int argc, wchar_t **argv)
{
  printf(L"Hello, world!\n");
  return 0;
}

And the Makefile:

Code: Select all

TARGET = helloworld.efi
 include uefi/Makefile

Compilation:

Code: Select all

$ make

That's it, everything is taken care for you, and you'll get helloworld.efi in no time!

Support is not fully POSIX compatible, the goal was not to create a libc, but to hide UEFI functions behind and interface that looks and works like POSIX. Despite of that and it's small size, there are quite a lot of convenience functions: wide-char to UTF-8 and vice versa converters, locatime() to get the current time, fopen, fread, fwrite, fprintf etc. The standard getchar() returns UNICODE codepoints in wchar_t too. Take a look at the documentation.

Known limitations: the build environment can handle any architecture, the lib is multi-platform too, but there's only x86_64 crt for now.

As usual, MIT licensed, Free and Open Source Software, without any guarantee and the hope that it will be useful. If you find any issues with it, please use the Gitlab's issue tracker.

Cheers,
bzt

[PeterX]

I want to have it all _my_ code and all from lowlevel upwards. So I think this is not the right thing for me.

But perhaps it could help the many people who ask for BIOS interrupts on UEFI PCs. Not by emulating BIOS, but by giving an easy API/ABI. That would be really great!!

Greetings
Peter

[nexos]

That's a pretty clever idea! I personally like the UEFI API, and somewhat detest POSIX in a way, but I'm sure many peoples are different. One thing, though. Your repo uses the UEFI symbol. Is that copyrighted?

[vvaltchev]

Cool idea, man! The only limitation I see is that the POSIX functions you implemented are just a tiny part of a UEFI bootloader.
90% of the time we still have to write code like:

Code: Select all

   status = fProt->Open(fProt, &fileHandle, filePath, EFI_FILE_MODE_READ, 0);
   HANDLE_EFI_ERROR("fileProt->Open");

   fileInfoBufSz = sizeof(fileInfoBuf);
   status = fProt->GetInfo(fileHandle, &gEfiFileInfoGuid, &fileInfoBufSz, nfo);
   HANDLE_EFI_ERROR("fileProt->GetInfo");

How can your library completely save us from using UEFI's protocols, guids etc?
A crazy idea could be to go much further in that direction and implement a whole sysfs-like in-memory filesystem and expose the UEFI objects there somehow.
So, instead of opening protocols we end up opening "files". But, that very likely won't be that better as an interface, because files and directories support a limited
set of operations, while UEFI objects are pretty complex sometimes.

Maybe you could re-design the whole UEFI interface with a fresh and friendly C++ interface or something like that using the POSIX functions you have as a starting point. I don't know, just think about that.

Vlad

[PeterX]

I'm not sure if the suggestion from vvaltchev are going in the same direction as your project...

Greetings
Peter

[bzt]

I want to have it all _my_ code and all from lowlevel upwards. So I think this is not the right thing for me.

What do you mean?

but by giving an easy API/ABI. That would be really great!!

That's exactly what POSIX-UEFI is about! Plus it also configures your host compilers so that you can compile PE no matter what.

Cheers,
bzt

[bzt]

Cool idea, man! The only limitation I see is that the POSIX functions you implemented are just a tiny part of a UEFI bootloader.

What do you miss? There are functions to get keypress, print out strings, open, read and write files. There are also some additional interfaces defined for your convenience (Serial IO, Block IO, GOP, etc.), but if you lack something, I can add it.

90% of the time we still have to write code like:

Absolutely not! No more fProt->Open(), that's the point!

How can your library completely save us from using UEFI's protocols, guids etc?

For example the code above looks like this with POSIX-UEFI:

Code: Select all

FILE *f = fopen(filePath, L"r");

That's the whole point, if there's a libc equivalent, you can use that just like you would in any other POSIX system (and from developer perspective you absolutely don't have to worry about if FILE is actually a EFI_FILE_HANDLE struct or an EFI_SIMPLE_TEXT_OUTPUT_INTERFACE, the library figures that out for you). The next code also works as expected (and '\n' converted to '\r'+'\n' under the hood just as UEFI wants it):

Code: Select all

fprintf(stderr, L"Hey you! Don't help them to bury the light, don't give in without a fight!\n");

But yes, there are some UEFI functions which do not have libc counterparts (like setting the screen resolution for example), for those you still have to use GUIDs. In POSIX-UEFI, that goes exactly the same as in EDK2, for example:

Code: Select all

efi_guid_t gopGuid = EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID;
status = BS->LocateProtocol(&gopGuid, NULL, (void**)&gop);
gop->SetMode(0);

A crazy idea could be to go much further in that direction and implement a whole sysfs-like in-memory filesystem

You could, but that's not what POSIX-UEFI is about. It does not want to redesign the interface, it just ships a small and sane POSIX build environment configurator for the host OS and provides simple and easy to use POSIX-like wrappers for the application. That's all.

Maybe you could re-design the whole UEFI interface with a fresh and friendly C++ interface or something like that using the POSIX functions

About that last part, that's exactly what POSIX-UEFI does ;-P Hides all the GUIDs and everything it can behind POSIX functions (and yes, not everything can be hidden).

I want to have it all _my_ code and all from lowlevel upwards. So I think this is not the right thing for me.

What do you mean?

This might be confusing, so here's a little explanation POSIX-UEFI ships three files:

First, crt0.o, no matter how much you try, you'll need this to get your entry point called. You don't have freedom on how you set up the registers for the ABI.

The second one is libuefi.a, which provides functions like "malloc", "fopen" etc., of course you could write all those on your own, but there's not much point in it as all they do is calling UEFI functions (if you'd wrote it yourself, you'd have to write 99% exactly the same code). Because it's just a wrapper, it's small, less than 32k.

And finally the Makefile, which sets up the compiler flags, well, you'd have to use exactly the same flags if you want to compile for UEFI under Linux. But sure, you're welcome to do all the groundwork yourself

A little update
I feel like Ken Thompson, I was very productive, and with the latest update I've shrinked the codebase from 192K to less than 96K while I increased the features at the same time.

If someone misses a libc wrapper, UEFI functions can be called just as-is, like in the EDK2 without the need of "uefi_call_wrapper", no matter if the compiler is gcc or CLang. Throwing out "uefi_call_wrapper" reduced the source significantly.
The whole CLang compilation toolchain has been rewritten, so that it now generates PE object files and uses the M$ compatible lld-link (used to be ld.lld) to link them into a DLL. No more objconv trickery with LLVM any more!

Cheers,
bzt

[bzt]

That's a pretty clever idea! I personally like the UEFI API, and somewhat detest POSIX in a way, but I'm sure many peoples are different. One thing, though. Your repo uses the UEFI symbol. Is that copyrighted?

I don't know. I don't think so, but if Intel reaches out, I'll remove it ASAP. I just wanted to indicate that it's an UEFI project, hence the logo. For example if your product runs on Windows, and to indicate that for your customers you put a Windows logo on its box that is okay, and isn't a violation of the copyright. This is the same thing.

Cheers,
bzt

[thomtl]

You can find the guidelines for using the UEFI logo on their website, Link.

[bzt]

You can find the guidelines for using the UEFI logo on their website, Link.

Thanks, but that document only talks about how companies can use the logo on their products. The UEFI Forum are too stupid to think about anything else than big money corps selling software. Well, I'm no company and I ain't selling anything. And POSIX-UEFI isn't a product which runs on UEFI either, it's an alternative to UEFI for other products. Anyway, I gave it a thought and decided to change the color. That document makes it perfectly clear that the UEFI logo is red.

I've come to the conclusion this is much much better now. The resemblance of the shape expresses that it runs on the same environment, but the different color makes it clear that this is something different, not part of that usual UEFI bloatware and madness. Plus all around the world the symbolic of red is "don't do it", "don't go", while on the other hand in 99% green means "good", "okay", "you can go with it"

More updates
I've added feof, dirent, opendir, readdir, mkdir, rmdir, unlink, stat, fstat, mktime, time, rand, srand, getenv, setenv etc. It is still less than 128k, and the file operations are now fully covered.
Because UEFI firmware isn't actually a POSIX operating system, there are many limitations: dirent only knows about DT_DIR and DT_REG, and stat doesn't have st_dev and st_ino fields either. The st_mode supports S_IFIFO for stdin, stdout and stderr though. Using rand() is not exactly POSIX, because it uses EFI_RNG_PROTOCOL if it can, and only resorts to standard POSIX behaviour if that interface isn't found. I had to change getenv and setenv a bit, because UEFI environment variables are binary blobs.

I've also added ARM AArch64 support, but for now it only works with LLVM (GNU ld has some unresolved relocation issues with ImageBase. I still have to figure out a workaround for that).

With this all the features I originally planned are implemented. If anybody has any ideas what more UEFI functions should be added and what POSIX function should they be masquared with, let me know!

Cheers,
bzt

[Korona]

Logo usage requests should be emailed to [email protected].

The linked document makes clear to you should ask for permission using that email ^. Maybe omit that "The UEFI Forum are too stupid [...]" sentence from that email, though.

[kzinti]

The resemblance of the shape expresses that it runs on the same environment, but the different color makes it clear that this is something different, not part of that usual UEFI bloatware and madness. Plus all around the world the symbolic of red is "don't do it", "don't go", while on the other hand in 99% green means "good", "okay", "you can go with it"

The linked page makes it clear that you are not allowed to change the colour of the logo. Changing it doesn't make it a different logo or allow you to use it.

[vvaltchev]

For example the code above looks like this with POSIX-UEFI:

Code: Select all

FILE *f = fopen(filePath, L"r");

That's the whole point, if there's a libc equivalent, you can use that just like you would in any other POSIX system (and from developer perspective you absolutely don't have to worry about if FILE is actually a EFI_FILE_HANDLE struct or an EFI_SIMPLE_TEXT_OUTPUT_INTERFACE, the library figures that out for you).

OK, but how I can select the partition or the physical drive? Is there any special URL-like syntax or POSIX-UEFI supports only paths inside the current partition?

But yes, there are some UEFI functions which do not have libc counterparts (like setting the screen resolution for example), for those you still have to use GUIDs. In POSIX-UEFI, that goes exactly the same as in EDK2, for example:

Code: Select all

efi_guid_t gopGuid = EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID;
status = BS->LocateProtocol(&gopGuid, NULL, (void**)&gop);
gop->SetMode(0);

Yep, and there are many cases like that. For example? How can I access a disk or a partition as a binary blob? I'll have to use UEFI.
Or, how can I change the video mode, clear the screen, move the cursor etc. ? Still need to use UEFI directly.

You could, but that's not what POSIX-UEFI is about. It does not want to redesign the interface, it just ships a small and sane POSIX build environment configurator for the host OS and provides simple and easy to use POSIX-like wrappers for the application. That's all.

I understand that now. I just got inspired by your idea and I started thinking about the more general idea of wrapping UEFI with a different interface. At the moment, I believe that the POSIX model cannot map well UEFI, roughly because UEFI is not a UNIX-like kernel. While some POSIX functions might be perfect for UEFI, in other cases you will inevitably end up with limitations. The need to access other partitions and disks is one of them. On a UNIX kernel, we can use the special device files in /dev plus the POSIX primitives, here we cannot do that. We'd need maybe a different mechanism to enumerate all the disks and partitions. The same applies for video modes, network devices and other fancy UEFI features.

One way to approach this problem is to implement something like a sysfs and expose everything through it, keeping a POSIX interface on the top. But I'm not really a fan of this approach because it's too tricky, will have plenty of overhead and it won't be so simple to use. The other idea is to just propose a different model, exposing the whole UEFI with an alternative interface. Finally, the 3rd approach, yours, is to just offer some POSIX functions without pretending to cover the whole UEFI.

I get that, just I wanted to offer some perspective about the other approaches and maybe make you at least consider the idea of not being strictly POSIX-compliant. You could offer in your library a set of functions designed for UEFI which are just inspired by POSIX, not strictly following it.

[bzt]

The linked page makes it clear that you are not allowed to change the colour of the logo. Changing it doesn't make it a different logo or allow you to use it.

If they have an issue with that, they should let me know, and then, but not before I'll change that picture. Seriously, an icon for the repo is the least important thing for an interface wrapper. Its not that that logo is included in the source in any way, it's just an avatar on the gitlab's preferences page. If you do a git clone, you won't get the logo

OK, but how I can select the partition or the physical drive? Is there any special URL-like syntax or POSIX-UEFI supports only paths inside the current partition?

Okay, one more time: POSIX-UEFI is a wrapper around UEFI calls. If UEFI itself can't access paths inside a partition, then POSIX-UEFI can't either.

Yep, and there are many cases like that. For example? How can I access a disk or a partition as a binary blob? I'll have to use UEFI.

What do you mean "many cases"? So far GOP is the only one that I couldn't wrap up in POSIX calls. Everything needed by a basic boot loader covered.

Or, how can I change the video mode, clear the screen, move the cursor etc. ? Still need to use UEFI directly.

What is your problem? By default you have to use that messed up UEFI interface for EVERYTHING or use those braindead library functions from EDK2 (which do not cover every UEFI protocol either btw). When you use POSIX-UEFI, you can spare yourself and use a much simpler interface in 99% of the time. And even for those 1% native UEFI calls POSIX-UEFI provides a simple implementation, a simple and fast build system (compared to EDK2's bloat). What is that you can't understand about this? If you are your own enemy, then go ahead and use that horrible EDK2. I've done that, I know exactly what I'm talking about. No sane people would ever think EDK2 is good or at least a bit usable.

Let me give you an example, under Linux, your application simply gets the command line arguments on the stack. Under UEFI, you need a helper function, ShellCommandLineParse which requires an additional library, ShellPkg, which in turn querying the EFI_SHELL_PARAMETERS_PROTOCOL and if that fails then the SHELL_INTERFACE_PROTOCOL... That is, if you're using the EDK2, because there's no ShellCommandLineParse in GNU-EFI, it calls that same function GetShellArgcArgv. And that's because EDK2 constantly renames libraries and functions breaking compatibility. They used to have StdLib with LibC and PosixLib, and now they don't. Should I continue how ugly UEFI actually is? Terrible ABI wrapped up in everchanging libraries... Yeah, what could go wrong?

GNU-EFI is much better, but it has its own pitfalls (the need of uefi_call_wrapper and doesn't compile with CLang).

At the moment, I believe that the POSIX model cannot map well UEFI, roughly because UEFI is not a UNIX-like kernel.

Think it again. Apart from the ABI their API is not so different. UEFI mimics Windoze calls, and you can convert those into libc (anybody who has written a multiplatform application knows that). The only real difference is, that UEFI needs a huge load of helper functions to hide the actual protocol interfaces, while a true POSIX kernel doesn't, just a single libc with well-defined interface to wrap up syscalls.

While some POSIX functions might be perfect for UEFI, in other cases you will inevitably end up with limitations.

For those rare cases you use the same interface that you'd have to use for EVERYTHING otherwise.

The need to access other partitions and disks is one of them. On a UNIX kernel, we can use the special device files in /dev plus the POSIX primitives, here we cannot do that.

Yes, we can! Here are the list of pseudo device files under POSIX-UEFI:
/dev/stdin - returns EFI_SIMPLE_TEXT_INPUT_PROTOCOL instance (ST->ConIn)
/dev/stdout - returns EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL instance (ST->ConOut)
/dev/stderr - returns EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL instance (ST->StdErr)
/dev/serial - returns EFI_SERIAL_IO_PROTOCOL instance
/dev/diskN - returns EFI_BLOCK_IO_PROTOCOL instance
...anything else - returns EFI_SIMPLE_FILE_SYSTEM_PROTOCOL instance

We'd need maybe a different mechanism to enumerate all the disks and partitions. The same applies for video modes, network devices and other fancy UEFI features.

No, we don't. You're mixing up interface with implementation. Two totally different implementations might have the same interface. A good example fprintf. That's the best thing in UNIX and POSIX, everything is a file.

I get that, just I wanted to offer some perspective about the other approaches and maybe make you at least consider the idea of not being strictly POSIX-compliant. You could offer in your library a set of functions designed for UEFI which are just inspired by POSIX, not strictly following it.

Never intended to by strictly POSIX-compliant. That's not possible without a kernel. The point is, use interfaces that look like POSIX so that people don't have to learn and s*ck with that horrible GUID interface and learn nearly a hundred of helper functions from the supporting library (which is constantly changing btw). Instead programmers can use a few functions that they are already familiar with, so there's no need to learn everything from the gourd up.

One example, which is more obvious to you? Which one should an UEFI newbie choose?

a) find the system table, find boot services in it, and LocateProtocol method in it. Then use that method with some magic numbers to return a struct which in turn must be defined exactly the same way in your application as in the firmware, and which should contain a method called GetRNG() which has parameters that you have to learn

- or -

b) call the well-known rand() that you're already familiar with to do the same thing.

Cheers,
bzt

[bzt]

The linked page makes it clear that you are not allowed to change the colour of the logo. Changing it doesn't make it a different logo or allow you to use it.

I stand corrected. That link does not say that you're not allowed to change the color. But it does say that changing the color makes it a different logo:

Code: Select all

The UEFI logo may only be used or printed in one color - PMS Red 032 U.

So using a different color means that's not the official UEFI logo any more, and their terms do not apply to my repo's green avatar, period.

Cheers,
bzt