OSDev.org

I am looking at adding support for "#!" execution, and am considering the pros/cons of implementing it in kernel versus user space. There is one obvious bit of overhead doing it in user space, which is an additional system call (*). However, this alone does not seem significant enough to justify a kernel implementation.

For folks who have implemented this or know more about it, what are your thoughts? There seems to be a fair bit of variation across operating systems (this is the most detailed source of information I have found).

(*) This is assuming an implementation where we try to run the file as an executable, and if that fails, check to see if it's a script and then call the interpreter. Alternatively, we can do the check first and avoid the additional system call, but that will add overhead to launching all executables, which seems worse.

Typically, part of the "run this file as an executable" logic is a magic check to determine both if the file is an executable and what kind of executable it is.

ELF has four magic bytes: 0x7F, 'E', 'L', 'F'. Dynamic ELF executables need to be run through a loader (conveniently called an interpreter), so you may need to load a different executable and run it in order to run an ELF.

DOS and PE executables have two: 'M', 'Z' (hence the pre-PE executables are called MZ executables).

You can implement hashbangs by treating '#', '!' as two magic bytes, and then load the requested executable just like you would a dynamic loader for an ELF.

That executable might itself be another hashbang, or an ELF with a dynamic loader, so you need to support this process being recursive - but generally with a limit (which is why execve lists ELOOP in the manual for possible errors).

Thanks. I have previously implemented dynamic loading along the lines you described. And this is pretty much how I would add hashbang support in the kernel. But I'm wondering what is the benefit of doing that versus in libc? Is it just to avoid an additional system call, or is there some aspect to the functionality that cannot be handled in user space?

As an aside, it seems that Linux is one of the very few Unix-like operating systems that support nested hashbangs (according to this). The POSIX spec only indicates the ELOOP return code from execve for too many symbolic links in the path (not surprising since POSIX doesn't specify hashbang behavior).

yr wrote:But I'm wondering what is the benefit of doing that versus in libc? Is it just to avoid an additional system call, or is there some aspect to the functionality that cannot be handled in user space?

From the perspective of POSIX (which may not specify hashbangs, but it also doesn't specify binary formats at all, so a hashbang script is no less valid as a program than a compiled C binary), there is no distinction to be made here (assuming your execve is where this support exists), but you may have practical reasons to want this functionality in a kernel instead of in a libc.

Maybe you want to support suid hashbang scripts (can you do that at all in a libc? it's difficult to do securely even in a kernel), maybe you want to be able to add support for more magic interpreters in the future (do you support statically-linked binaries? one with an outdated libc might not be able to execute some newly-supported executable in the future), maybe you want to be able to add new magic interpreters at runtime (like Linux does with binfmt_misc; doing this in a way that a libc can be made aware of them would be annoying - maybe a file in /etc that gets read at startup? but what if my application is already running, do I need to re-read it on every exec?). Similar to the topic of static linking, some modern languages have opted for direct system call interfaces instead of relying on a libc - would your support patch for those also include porting the logic for hashbangs?

yr wrote:But I'm wondering what is the benefit of doing that versus in libc?

The kernel already has to open the file to figure out what to do with it. libc doesn't. The kernel is in a way better position to figure all of this stuff out. According to POSIX, execl(), execle(), execv(), and execve() are supposed to be async-signal-safe, and that means they cannot allocate memory. But changing the command line from {"./file"} to {"sh", "./file"} requires one pointer more. You could allocate the memory on stack, but at least with execv() and execve() you risk a stack overflow in that case, especially if an attacker gets to control the argument array size.

The kernel on the other hand is in a perfect position to recognize the executable format. It has to do so anyway when asked to execute a file, and it can allocate memory for the arguments because it is in a normal syscall context.

yr wrote:But I'm wondering what is the benefit of doing that versus in libc?

If I recall correctly, and following a suggestion from the #gcc IRC channel, I had to include support to execute scripts in sys_execve() (using the shebang mechanism) as it seems to be a POSIX requirement when compiling GCC natively.

Thanks for the responses. There seems to be enough here to make a kernel hashbang implementation preferable (future enhancements, signal safety, etc.). It is interesting though that while POSIX doesn't specify anything around hashbang execution, it does say the following:

In the cases where the other members of the exec family of functions would fail and set errno to [ENOEXEC], the execlp() and execvp() functions shall execute a command interpreter and the environment of the executed command shall be as if the process invoked the sh utility using execl() as follows:

execl(<shell path>, arg0, file, arg1, ..., (char *)0);

where <shell path> is an unspecified pathname for the sh utility, file is the process image file, and for execvp(), where arg0, arg1, and so on correspond to the values passed to execvp() in argv[0], argv[1], and so on.

This seems very much along the lines of "try to run it as an executable, and if that fails, try to interpret it as a script". This is why I was considering a hashbang implementation in user space as just a generalization of the above behavior.

yr wrote: This seems very much along the lines of "try to run it as an executable, and if that fails, try to interpret it as a script". This is why I was considering a hashbang implementation in user space as just a generalization of the above behavior.

Yeah, but that only allows for shell execution. There are other script interpreters. Also, due to security concerns, there are a couple of libcs that don't implement this requirement.

Of course, these functions don't have to be signal safe, so they could also just allocate the memory. Hmmm....

We could take a page from Window's book here. Have your loader support multiple executable formats in a generic manner, one being ELF and the other hashbang. E.g, we could have sys_execve actually look at a function table, with functions each format. So sys_execve would (indirectly) call elf_check_file, and if that failed call hashbang_check_file. Of course it would be looping through a function table instead of directly invoking the functions. When one succeeds, it in turn would call (e.g.) elf_load or hashbang_load, again indirectly.

Now one may think that sounds overcomplicated, but I think it's the most elegant way to handle this issue. I believe Windows operates (sort of) similarly, as it has the ability to launch .exe's, .pif's, .bat's, and so on.