Issue with bootloader

[nexos]

Hello,
I am currently working on a bootloader written in C and Assembly that is designed for floppies. As I am using PE, it being built under Visual Studio. I was wondering what the best way to make a floppy driver for a bootloader would be. If it is to drop down to real mode, how would I link that into 32 bit PE application?

[iansjack]

Why not do the bits involving reads from the floppy before you enter Protected Mode? That way you can use BIOS calls.

[BenLunt]

Your question is a little vague.

I think @iansjack took it as you have already created a boot loader which has loaded another stage/kernel file and you want to continue to read from the floppy.

I took your question as you are writing the actual boot sector using Visual Studio and wondering how to do it.

Which one of us is correct?

If the former is correct, then are you looking to write a driver for the floppy drive to read and write from/to the disk? As @iansjack stated, you would be better off reading all you need to read using the BIOS before moving to protected mode. However, writing a driver for the floppy drive can be interesting and enjoyable to do. I have done it.

If the latter is correct, you need to forget about Visual Studio and C and stick with assembly to write the boot loader. The later stage loader and kernel can be written in Visual Studio as a PE file, but the (1st stage) boot loader must be assembly.

Ben
- http://www.fysnet.net/media_storage_devices.htm

[nexos]

I have decided to load all files before entering protected mode. Thank you for your help

[neon]

Hi,

Interestingly, our boot loader was written originally in MSVC. What we did was implement a separate function, ioServices, that drops the system down to real mode, calls the BIOS, and returns back to protected mode. Higher level code can then call it at any time to call BIOS interrupt services. We provided this function some time ago but never announced it publicly. It can be seen here. The idea was inspired off of the old bios.h geninterrupt call. It is assembled & linked as any ordinary object file. It is important to note that NBOOT_IMAGE_BASE and NBOOT_IMAGE_SEGMENT is dependent on the in-memory address of the boot loader. Of course, the boot loader and any buffer must be accessible from real mode (boot loader must be linked >= 64k base address.)

For completeness, here is our disk read method for floppy drives which demonstrates using this.

Code: Select all

PRIVATE unsigned int DiskReadSectors (IN unsigned int cylNumber, IN unsigned int sectorNum, IN unsigned int  headNum,
				 IN unsigned int  driveNum, OUT unsigned char* buffer) {

	INTR     in;
	INTR     out;

	if ( buffer == NULL )
		return 0;

	/* int 0x13 function 2. */
	_ah (in.eax) = 2;
	_al (in.eax) = 1;	/* numSectors */
	_cl (in.ecx) = (unsigned char) sectorNum;
	_ch (in.ecx) = (unsigned char) cylNumber;
	_dh (in.edx) = (unsigned char) headNum;
	_dl (in.edx) = (uint8_t) driveNum;
	in.es        = SEG    ( (uint32_t) buffer);
	_bx (in.ebx) = OFFSET ( (uint32_t) buffer);

	/* call BIOS. */
	io_services (0x13, &in, &out);

	/* status code is in AH.  */
	return _ah (out.eax);
}

[BenLunt]

Hi neon,

I use a very similar technique in my boot code. (Here is an older version) It is called from unreal mode, allowing the caller to have memory, stack, and code to be anywhere in the 4gig address space. The call, however, requires all memory used by the service to be accessible by a real mode BIOS.

An alternative to your technique, I use the same space for the 'in' registers as I do for the 'out' registers. I don't see any reason to preserve the 'in' state.

Also, I don't worry about calling it after the loader transfers control to the kernel. Once the kernel takes control, the loader is no longer a part of the system. However, I see an advantage to your technique.

In theory, since I use unreal mode through out my loader, I wouldn't need something like this. A real mode BIOS should work just fine in unreal mode. However, with a technique like the one I use, my loader, its stack, and any memory buffer needed, can be anywhere in the 4 gig space, using all of the 32-bits of a register, and still call a BIOS service call without worries.

Another note, my code uses self modifying code just like yours:

Code: Select all

        mov   ebx, .call_service - NBOOT_IMAGE_BASE
        mov   byte [ ebx + 1 ], dl
    .
    .
    .
.call_service:
        int   0

One problem that I know is that you have to make sure there is enough of a code stream between the self modifying code and the self modified code so that the processor hasn't already loaded that code into the cache. Granted on some systems, the processor will see this and compensate. Older machines may not. However, I think you have plenty of space between the two.

I wanted to use a serializing instruction, such as CPUID, to guarentee this, but this assumed I had checked that the CPUID instruction was first available before a call like this. The chicken and the egg scenario here is that I use this "wrapper" to call the BIOS to print a message to the screen. If a CPUID instruction is not found, I would want to print an error message stating so. However, to print the error message, a CPUID instruction would have to be available to do so. Chicken/egg...

Anyway, good point. I think a good loader needs a BIOS wrapper like this.

Ben

[neon]

Hi,

The core boot loader program running in 32 bit protected mode or 64 bit long mode as opposed to unreal mode was a design choice to facilitate supporting BIOS and UEFI 32 and 64 bit builds; only calls to the IO services require memory accessible from real mode. Two memory pools are used by the allocator to facilitate buffers for real mode but it may be copied anywhere in the address space.

We did consider the CPU caching the instructions ahead before it gets to it. We were really trying to find a method to avoid self modifying code but alas after research in existing solutions this might be the only way without compiler support. I don't know if there is a more full-proof way for this, so am just waiting for it to potentially fail in which case we know we need to add something more.

The basic idea is the same though -- abstracting BIOS calls behind a single function. The details of how its done I believe is heavily dependent on design goals -- Unreal mode can be a good option for sure. For the chicken/egg scenario, perhaps a beep would work to indicate some kind of fatal error? Having the loader output debug records to the serial port for any attached debugger is also very useful here -- we don't use BIOS for that for obvious reasons.

[kzinti]

I wrote some code to call the BIOS from protected mode. It is self modifying code in that it modifies the "int xx" instruction to call the BIOS function I want. But not only is there a lot of code between the modification and the invocation, the processor also has to switch from protected mode to real mode. Surely this will flush the instruction pipeline. It would be rather surprising if the CPU was speculating real mode instructions while in protected mode.

There are also other ways beside using the CPUID instruction to flush the CPU pipeline.

[nullplan]

I wanted to use a serializing instruction, such as CPUID, to guarentee this, but this assumed I had checked that the CPUID instruction was first available before a call like this. The chicken and the egg scenario here is that I use this "wrapper" to call the BIOS to print a message to the screen. If a CPUID instruction is not found, I would want to print an error message stating so. However, to print the error message, a CPUID instruction would have to be available to do so. Chicken/egg...

I remember reading in the Linux source code once that on CPUs without CPUID support, a conditional jump will flush the instruction queue. Therefore you only need to test for CPUID availability and do a conditional jump past the CPUID to flush the I-queue there. And on machines with CPUID support the jump would not happen, but then the CPUID instruction would synchronize things.

Alternatively, mark the CPUID instruction with a symbol and add a #UD handler that recognizes when IP == that symbol, and then returns to the point after it.

Honestly though, why don't you just put a bunch of if-then-else chains in there? I mean, with interrupts 0x10 and 0x13, you already have covered most of what people ever want to use BIOS for, right? Alternatively, use macros to get two different functions that are almost identical, but end up calling int 0x10 in one case, and int 0x13 in the other. That sidesteps the whole self-modifying code issue altogether.

[kzinti]

I am fascinated by this self-modifying code issue. There is no issue. Just modify the int instruction at the beginning of your function that calls the bios. There is so much to do, including setuping and switching to real mode, that by the time you get to the int instruction, there are no concerns.

[Octocontrabass]

Modern x86 CPUs handle self-modifying code automatically, as long as the code is being modified and executed at the same linear address. (A serializing instruction is required if writing and executing will use different linear addresses.)

Old x86 CPUs require a taken conditional jump or an unconditional jump to flush the prefetch queue.

Intel and AMD have various recommendations for self-modifying code, but both agree that an unconditional jump will satisfy the requirements of all CPUs, old and new. Adding a single "jmp short $+2" instruction after the write is enough.

There is so much to do, including setuping and switching to real mode, that by the time you get to the int instruction, there are no concerns.

You're right, switching from protected to real mode involves an unconditional jump and a serializing instruction.

[nexos]

Thank you for your help! I have currently paused this project in favor of NexNix, a Unix like operating system, but I will definitely use your suggestion when I make it again.