OSDev.org

Posted: **Sun Sep 01, 2013 4:15 pm**

it's something i just thought about, i don't know if it would work, but i think it should

normally, most of the 16 bits intel opcodes can be run in 32 bit real mode, the biggest barrier is the memory model and how registers are used to represent address differ between the two mode

in 16 bit real mode , the segments register represent a physical memory address / 16, but in segmented 32 bit protected mode, values of es can be mapped easily also to physical memory address through the gdt, so normally it should be possible to arrange the gdt in sort that the address of the segment selector match value of the adress es mean in real mode

there is a little issue of granularity, because segment selectors are supposed to be ofset in the gdl table, so their valid values have a granularity of 8 , so value in es would have to be multiple of 8, so it mean a memory granularity of 128bytes as es is multiplied by 16 in real mode to have the physical address, but es is supposed to be a segment selector and should be used to store value of a about a segment (64k) / 16 , so about 4k, and it's not very likely es would be used with value in the range that the gdt granularity impose of valid segment selector values

there is also the issue of interupt, but if either the standard base pic should be enabled, or configuring any pic controler to route interupt vector stored in the real mode ivt

from there normally it should be possible to run real mode code directly from the 32 bits segmented mode no ? provided the memory address in the segment selector that represent the first 1Mo of physical memory are accessible ?

Posted: **Sun Sep 01, 2013 11:08 pm**

The problem is most real mode application assume things that is not friendly in protected mode. For example, hook ISR by directly manipulate the IVT, also note that GDT is limited to 8192 entries so you can't set ES to FFFF, which was possible in real mode.

Anyway, it may work with a lot of emulation and exception handling, but the v86 mode was invented for exactly such purpose.

PS: Also note that there is 16 bit protected mode.

Posted: **Mon Sep 02, 2013 9:00 am**

bluemoon wrote:The problem is most real mode application assume things that is not friendly in protected mode. For example, hook ISR by directly manipulate the IVT, also note that GDT is limited to 8192 entries so you can't set ES to FFFF, which was possible in real mode.

Anyway, it may work with a lot of emulation and exception handling, but the v86 mode was invented for exactly such purpose.

PS: Also note that there is 16 bit protected mode.

they are limited to 8192 entry, but each entry is 8 bytes, and es increase 8/8 for each entry, so it can theorically map the whole range

for the isr, the interuption handler could be mannaged to route interupt used by real mode directly to the vectors in the ivt

the goal would be first to be able to run bios funcs, or maybe with a bit more work to handle dos applications, without needing to do any mode switch at all

and it's not possible to share memory used in v86 mode with program running in protected mode is it ? i don't know how the cpu determine the memory used by the v86 mode, and what happen to the memory when you exit the v86 mode, and how a program could pass parameters or share a memory area with the code running in this mode

for the purpose of executing bios function, theorically just adding entry for segment 0xa000,0xB000,0xC000,0xD000,0xE000,0xF000, should already cover most of the segment registers value a bios code could use, i guess if an invalid segment register is used, it should throw an exeption so error shouldn't be to hard to track

assuming the os use the linear framebuffer address, and a 16 bit program like a dos application, use the vga address, it could be possible to make in sort that vga frame buffer address are remapped somewhere else anywhere in the whole Go memory , and then used as a window backbuffer to run dos application easily in windowed mode without too much problem at all

for 32 bit dos program, it would probably be harder cause they are supposed to use the same ressources than the 32 bit os would use, so it would be more problematic

Posted: **Tue Sep 03, 2013 1:55 pm**

i tried to code this but it's not as easy as i thought

i try to execute bios code from protected mode to see if that can work, but i've encountered some little issue with it

first some area need to be accessed both as code and data segment, as i use segment selector mapped to an address it's not possible to map the same address as code and data in the same time, i solve this for now by giving a special mapping to code segment it should be ok as long as the 16 bit code doesn't do a far jump using a segment selector that need to be used also as data segment

bios code also has to be executed from a 16 bit code segment, so it make the managment of the stack rather tricky regarding interupt and exeptions, as the instruction int/iret behave differently depending if it's 16 bit or 32 bit

i looked into intel manuals and various sources, it seem the behavior of int/iret instruction is rather complex, in the intel manual the pseudo code of the instruction is several page long and it depend on lot of parameters, depending if the cpu is in real mode or protected mode, and there are implicit 16 bit and 32 bit version of the opcode that the assembler choose depending on the [bits ] directive

as i have to make the call to the 16 bit code that is supposed to be an interuption handler from the 32 bit code, the stack must be prepared as such as the 16 bit version of the iret opcode can return, and so the call/jmp/int cannot be made using the 32 bit opcode

the stack also need to be setup to be used with the 16 stack pointer (sp), so the stack pointer also need to be prepared to be able to be used with the 16 bit ofset register

for now it seems to work a bit, i tried to set video mode, and it seem to works to a degree, in the sense it seem to switch the mode, and it make use of the segment register properly, but it throw a page fault at some point that i'm trying to debug

i made some special interuption handler that lookup for the entry in the ivt to make a jump to it

Code: Select all

;--------------------------
%macro interupt_hndl 1

interupt_%1:

[.....]
	movzx	ebx	,	word [0x0+(%1)*4]                    ; ofset part of the ivt vector
	movzx	edx	,	word [0x0+(%1)*4+2]                ; segment part of the ivt vector
	shr		edx	,	4                                               ; trick to match code segment selector to address >> 4

		
	mov ax				,0x48                                 ;prepare the stack for 16 bit code
	mov ss				,ax
	mov esp				,0x8000
		
	pushf
	push	word  cs                                                             ;push value in the stack for the iret, as it's a 16 bit iret, the stack cannot be prepared using a regular int call
	push	word end_of_real_interupt_%1
	
        mov ax,13h                                                                 ;set vga mode 13h

	push	word  dx                                                             ;push the segment selector/address of the ivt vector code segment
	push	ebx                                                                    ;push the ofset of the of the ivt vector
	db 0xCB                                                                     ;32 bit far ret opcode , which load the ivt vector into cs:eip

the gdt is setup like this

Code: Select all

		seg_access	=GDT_ACCESS_PRESENT|GDT_ACCESS_WRITE|GDT_ACCESS_SYSTEM;
	seg_code_ax	=GDT_ACCESS_PRESENT|GDT_ACCESS_WRITE|GDT_ACCESS_SYSTEM|GDT_ACCESS_CODE;
	flags		=GDT_ENTRY_AVL;//|GDT_ENTRY_32bit;
	

	//set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x0,0x0,0xFFFF,seg_access,flags);

	
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x0C00,0x000C0000,0xFFFF,seg_code_ax,flags);                       ;code segment selectors for bios aera
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x0F00,0x000F0000,0xFFFF,seg_code_ax,flags);

	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x0040,0x00000400,0xFFFF,seg_access,flags);                           ;segment selectors for bios base address (0x40:XX)
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x9FC0,0x0009FC00,0xFFFF,seg_access,flags);                           ;segment selectors for bios edba
	




	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xA000,0x000A0000,0xFFFF,seg_access,flags);                             ;data segment selectors for bios aera
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xB000,0x000B0000,0xFFFF,seg_access,flags);
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xB800,0x000B8000,0xFFFF,seg_access,flags);
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xC000,0x000C0000,0xFFFF,	seg_access,flags);
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xD000,0x000D0000,0xFFFF,seg_access,flags);
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xE000,0x000E0000,0xFFFF,seg_access,flags);
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xF000,0x000F0000,0xFFFF,seg_access,flags);

	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x1F90,0x00001F90,0xFFFF,seg_access,flags);                ; entered manually, pointer to stack stored in segment by the bios 
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x1490,0x00014900,0xFFFF,seg_access,flags);                ;pointer to output data stack stored in segment by the bios 

	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x0100,0x00001000,0xFFFF,seg_access,flags);                ; space used for 16 bit stack
	set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x1000,0x00010000,0xFFFF,seg_access,flags);                ; space used for 16 bit ofset from kernel base address (0x10000)

	set_gdt_ptr_c(new_gdt->limit,new_gdt->addr);

/*
set_gdt_entry(new_gdt,&new_gdt_entries_ref,0xA000,0x000A0000,0xFFFF,seg_access,flags);

0xA000 => segment selector ofset in the gdt
0x000A0000 => physical memory
0xFFFF => limit
seg_access => gdt access
flags => gdt flags
*/

so i can just call int 10h from protected mode, and getting the 16 bit bios coded executed from this interupt handler, it would need to save registers to be passed to the interuption, but i'll do this latter, it seem to execute and change the mode

but it end with a page fault exception, from what i gather from the stack, it seem to be fired from within the bios segment, as cs and ds are set properly, but the calling stack seem to be setup as for a 32 bit exeption, albeit it's supposed to be fired from a 16 bit code segment, i couldn't really understand clearly if what change the behavior of int/iret regarding the stack is determined from the 16/32 bit attribute of the code segment, or if it's only cpu flags regarding real mode/protected mode, so i need to dig that more, or if in the absolute any version of the opcode could be used from any type of segment, as a 16 bit opcode could be present in the 32 bit segment, and a 32 bit opcode from a 16 bit code segment, or if the cpu determine which version of iret will be used in the exception handler only from general protected mode flag

i will debug this more to see if i can determine clearly the behavior that is expected from an interupt/exeption handler and an interupt call regarding the setup of the calling stack when the call have to go accross 16/32 bits protected mode segments

with an exception throw, the page fault exception could be caugth and the execution could still be resumed after the fault and apparently the video mode is set, but i'll try to figure was is going wrong

maybe i should put the interupt handler in [bits 16] section with nasm, the protected mode interupt controller should be able to switch the segment selector to the 16 bit code segment, and prepare the interuption/exeption calling stack for a 16 bit iret, which could make the calling context more compatible with the 16 bit code to be called, or i need to setup all the calling stack properly from the 32 bit code for the calling, and dealing with handler that can potentially be called from a 16 bit code segment

Posted: **Tue Sep 03, 2013 4:00 pm**

Sorry, I didn't read your post to the very end but I just want to say that BIOSes often use temporary switch to protected mode, so trying to run BIOS code from 32-bit PM is likely to either fail (not ring0) or crash (ring0) your PM setup.

Posted: **Tue Sep 03, 2013 4:09 pm**

Nable wrote:Sorry, I didn't read your post to the very end but I just want to say that BIOSes often use temporary switch to protected mode, so trying to run BIOS code from 32-bit PM is likely to either fail (not ring0) or crash (ring0) your PM setup.

it will be in ring0, and i guess a certain number of things should be checked when the bios return, it can setup protected mode, then either screw thing but for the moment i'll assume it's not and trying to find a way to figure out if it did, but the worst is that it should then restore the cpu to real mode before the return so that would need to be checked as well, i have some idea to handle the case

but it would be more to allow it to be used from a special kind of application context anyway , i don't plane to really mix the regular multi tasked application framework with bios call, but more to allow some special mode of execution where the computer can be programmed as if it was in real mode or more like from something like a dos extender with more or less exclusive access on all the resources

like having a special mode of execution that can call bios interuption with a flat memory model with an access that is assumed to be either exclusive or potentially dangerous for the system, but it could still run in most case even along side of the normal framework with a bit of care

if i can catch exceptions properly, it can still screw things up but not totally crash normally, but it's not supposed to be 100% fail safe bullet proof

Posted: **Wed Sep 04, 2013 5:00 am**

ok so for those it might interest, i mannaged to make this work, it seems to work fine, the video mode is setup and it return properly, even with 3 reentrant call to the interupt handler from the bios, without any switch to real mode

the interupt handler looks like this

Code: Select all

;--------------------------
%macro real_mode_interupt_hndl 1

real_mode_interupt_%1:

	pushfd								;	push cpu flags from caller
	pushad								;	push registers from caller
	mov ebp , esp						;	save stack frame to saved registers location
	push ds								;	push segment registers
	push es	
	
	mov			ax,0x10					;	set segments selector to segment starting at memory 0x000
	mov			es,ax
	mov			ds,ax
	
	cmp dword [protect_mmx_regs],0		;	save mmx in case bios use mmx or fpu
	je irq_hndl_no_mmx_%1
		call save_mmx_regs
		emms
	irq_hndl_no_mmx_%1:	
	
		

	;---------------------------------------------
	;start of the interupt routing
	;---------------------------------------------
	movzx	ebx				,	word [0x0+(%1)*4]				;	get real mode interupt vector ofset from ivt
	mov [real_mode_ofset]	,  ebx
		
	movzx	ebx				,	word [0x0+(%1)*4+2]				;	get real mode interupt vector segment from ivt
	shr		ebx				,	4								;	set the segment selector to (addr>>4) for code selector
	mov [real_mode_seg] 	,  ebx
	
	call  setup_registers_from_stack							;	set caller registers from the stack
	
	push dword [ebp+32]											;	set caller cpu flags from the stack
	popfd
	
	push  word [ebp+32]											;	setup the stack for 16 bit iret
	push  word 0x50												;	return address segment selector starting at kern base
	push  word (end_of_real_interupt_%1-kern_base)				;	return address ofset from kern base
	
	push  dword [ebp+32]										;   push caller flags on the stack
	push  dword [real_mode_seg]									;	push real mode interupt vector address on the stack
	push  dword [real_mode_ofset]				
	
	push eax													;	restore caller segment selectors from the stack
		mov ax			,	[ebp-8]
		mov es			,	ax
		mov ax			,	[ebp-4]
		mov ds			,	ax
	pop eax
	
	db		0xCF												;	opcode for 32 bit iretd, set cp:eip and cpu flags from the stack
	
	end_of_real_interupt_%1:
	
		
	pushfd														;   store cpu flags from the interupt on the stack
	pop dword[ebp+32]
	
	pushfd														;   store cpu flags from the interupt on the iret return stack
	pop dword[ebp+32+12]
	
	jmp 0x08:end_of_real_interupt_2_%1							;	restore code segment selector to 0x08
	end_of_real_interupt_2_%1:
	
	
	call setup_stack_from_registers								;	store registers returned from the interupt on the stack
	
	mov			ax,0x10											;	restore segment selector to 0x10
	mov			es,ax
	mov			ds,ax
		
	cmp dword [protect_mmx_regs],0								;	restore mmx registers
	je irq_hndl_no_mmx2_%1
		call restore_mmx_regs
	irq_hndl_no_mmx2_%1:

		
	lea esp				,	[ebp-8]								;	setup the stack to last push location
		
	pop es														;	restore segment registers to caller state
	pop ds
	
	popad														;	restore registers from the stack to state saved from interupt return
	popfd														;	restore cpu flags from state saved from interupt return

iret															;	return from interupt

%endmacro

then bios fuction can be called directly from 32 bit protected mode like this

Code: Select all

_read_bios_disk_geom_c:
	
    mov eax				,[esp+4]					;drive number
    mov [drv_num]		,al
    
    mov eax				,[esp+8]					;target addr
    mov [disk_infos_ptr],eax

	
	pusha
	mov  ebp,esp
	
	cli

	mov eax					,	0x0100				;setup the stack for 16 bit use (usable with 16 bit stack pointer)
	mov ss					,	eax
	mov esp					,	0x1000
	
	mov eax					,	0x00				;reset 32 bit register to zero
	mov edx					,	0x00
	mov ebx					,	0x00
	mov ecx					,	0x00
	mov esi					,	0x00
	mov edi					,	0x00
	
		
	
    xor ax		,ax									;call interupt 13h ah=0 (reset drives)
    mov dl		,[drv_num]
    int 13h
    jc bios_disk_geom_error
	
		        
	;-----------------------------------------
	;test for lba device
	;-----------------------------------------
	mov ah,41h									;call interupt 13h ah=41 (lba extension present)
	mov dl,[drv_num]
	mov bx,0x55AA
	int 13h
	jc no_ext_lba
	cmp bx,0xAA55
	jne no_ext_lba
	test cx,1
	jz no_ext_lba
		
		;-----------------------------------------
		;get lba sector size
		;-----------------------------------------
		
			
		
		mov ah			,	0x48									;call interupt 13h ah=48 (get extended infos)
		mov dl			,	[drv_num]
		lea esi			,	[shared_data real_mode_to_addr]			;setup buffer ofset from start of the kernel
		
		mov	ebx			,	0x1000									;setup segment selector to start of the kernel
		mov es			,	ebx
		mov ds			,	ebx
		xor ebx			,	ebx
		
		mov word [esi]	,	0x1A									;set the size of the output buffer (1A = extension informations version 1)
		
		int 13h														;call the interupt
		jc bios_disk_geom_error
		
		mov	ax					,0x10								;set segment selector to default (starting at address 0x000)
		mov ds					,ax
		mov es					,ax		
		
		mov edi					,[disk_infos_ptr]					;set pointer to buffer where the infos need to be copied
		lea esi					,[shared_data]					;set pointer to the information returned from interupt
		
                [.....]                                                                                                 ;copy the info     

	bios_disk_geom_done:
	
	
		mov ax					,	0x18                                       ;restore stack and segment selectors
		mov ss					,	ax
		mov esp					,	ebp
		
		mov	ax					,	0x10
		mov es					,	ax
		mov ds					,	ax
		
		
			
		popa                                                                                     ;return
		mov eax,1
	ret
	bios_disk_geom_error:
	
		mov ax					,	0x18                                   ;restore stack and segment selectors
		mov ss					,	ax
		mov esp					,	ebp

		mov	ax					,	0x10
		mov es					,	ax
		mov ds					,	ax		
	
		popa                                                                                                         ;return
		xor eax,eax
	ret

real mode can use null value for segment register which is illegal in protected mode, but they can be set to a valid segment selctor that point to the address 0x00 in the exeption handler if they have a null value and the execution can be restored at the same instruction

i can also catch exception thrown when an invalid segment selector is used, the return address of the exception handler point to the opcode, there is a single instruction that allow to set segment registers (opcode = 0x8E), and i can find the value that is attempted to be set in them from the instruction source register index, so i could make a system to automatically add the value in the gdt, and restore the execution, or i can easily add it manually when the code throw a page fault exception on this opcode

same system could be used to track access to cr0, the code segments used for real mode could be set with some privilege and then the unwanted instructions could be skipped or fixed from the exeption handler, and the execution resumed either after the instruction attempting to change cr0 or with fixed values for registers used as source operand

for now it seem to be able to work fine, it just miss a little something to pass on the cpu flags from the return of the actual call to the return from the interupt handler, but for now it works so yay =)

technically from there, bios functions could be used with memory area anywhere in the main memory, provided the good selectors are set in the gdt for the segments used

Posted: **Thu Sep 05, 2013 12:50 pm**

ok so now it works perfectly, can run all bios funcs from int 13h and int 10h without problem from protected mode =)

there is another trick to be carefull about, the bios might use the area into 0x40:XX , it's the bios reserved area, and it will use segment selector 0x40, as it is a low segment selector, it's most likely to be used by the system, this one was hard to figure out because it didn't cause any segfault as it is a valid segment selector but it need to be sure it point to the right place, at address 0x400, need to also add a selector for the bios edba

it still need a good exeption handler to catch some problems, mostly due to the opcode 0x8E to set segment selector value, or to catch invalid null segment register

Code: Select all

%define MOVE_TO_SEG_OP_CODE byte 0x8E

register_stack_idx_pusha:db 28,24,20,16,12,8,4,0

%macro exeption_hndl 1

exeption_%1:

	cli
	PUSHAD                                              ;save caller infos on the stack
	mov			ebp,esp
 	push 		ds
	push 		es
	
	mov			ax,0x10                      ;setup valid segment selector
	mov			es,ax
	mov			ds,ax

%if %1 == 0x0D	;page fault exe

        lea esi		,[ebp+32+4]               ;set esi to the start of calling frame (calling frame start at esp+4 due to an error code being pushed before ret address for page fault)

	mov eax,dword [ss:esi+4]						;calling address segment selector
	mov ebx,dword [ss:esi+0]                                          ;calling  address ofset

	cmp eax,0x0                                                            ; invalid segment address         
	je dump_exeption_call_frame_end
	
	test eax,0x07                                                           ; invalid segment selector
	jnz dump_exeption_call_frame_end

	mov		es	,	eax                                          ;fetch opcode at calling address
	mov		edi ,	ebx

	mov eax                            ,[es:edi]
	mov byte  [op_code]		,al

	cmp byte  [op_code],MOVE_TO_SEG_OP_CODE                   ;opcode is move to segment register
	je parse_op_code_mov_seg

	jmp parse_op_code_end_op
	
	parse_op_code_mov_seg:                                                   ;get the information from opcode rm/mod bits
		mov byte [op_rmod]		,1
		
		mov byte [op_src_reg]	,ah		;destination segment register
		and byte [op_src_reg]	,0x07				
			
		mov byte [op_dst_reg]	,ah		;source register
		shr	byte [op_dst_reg]	,0x03
		and	byte [op_dst_reg]	,0x07
		
		mov byte [op_mod]		,ah			;mod 
		shr	byte [op_mod]		,0x06
		and byte [op_mod]		,0x03		
		
		movzx	eax	,byte [op_src_reg]                                                      ;get source register value from the ones saved from caller on the stack
		movzx	eax	,byte [register_stack_idx_pusha+eax]
		mov eax		,[ebp+eax]
	
		mov [src_val],eax
	jmp parse_op_code_end_op

    parse_op_code_end_op:

    cmp byte [op_code],MOVE_TO_SEG_OP_CODE                                                        ;if value for source register is invalid, because of not multiple of 8
    jne exeption_not_seg_mv_%1                                                                               ;shl it by 4 and set the gdt entry for that value to the correct address

		movzx	edx  ,byte [op_src_reg]
		movzx	edx  ,byte [register_stack_idx_pusha+edx ]                                ;index from the pusha start address at ebp
		mov		eax   ,[ebp+edx  ]
		shl		eax   ,4
		mov		[ebp+edx  ],eax                                                                       ;set the updated value in the stack to be poped at return
	exeption_not_seg_mv_%1:
%endif

        lea esp ,[ebp-8]                                                              ;restore segment register from caller value
 	
	pop es	
	pop ds
	
	mov ax,ds                                                                        ;check for null segment selector, and set it to selector 0x10 starting at address 0x0
	cmp ax,0
	jne exeption_ds_ok_%1
		mov  ax,0x10
		mov  ds,ax
	exeption_ds_ok_%1:
		
	mov ax,es
	cmp ax,0
	jne exeption_es_ok_%1
		mov  ax,0x10
		mov  es,ax
	exeption_es_ok_%1:	
	
	popad                                                ;restore caller registers
	%if %1 == 0x0D	
		add esp,4                                   ;add esp 4 for the error code before the return address (from documentation)
	%endif
iret                               ;resume execution with updated registers value
%endmacro

with this exeption handler, attempt at setting incorect value in a segment register can easily be tracked and corrected, and for the moment it seems to works pretty fine with ll bios function i tested, and the bios use pointer to buffer in its stack in data segment register, so it compute manually seg/ofset value for an address on the stack, and it seem to resolve well, just need to shift invalid value that are not multiple of 8 attempted to be on the segment register, and setup the according entry in the gdt, and it works like a charm =)

would just need to make a system to see if the exeption is called from the bios to attempt at setting automatically new entry in the gdt for selectors that can be fixed in that way

i'll see if i can catch a bios function that enter protected mode to see how it can works

Posted: **Thu Sep 05, 2013 2:38 pm**

Oh, I'm really impressed now by how you overcome all these difficulties. Cool work, really!
Although personally I prefer V86/SVM/VMX/software emulator. And my dream is to experiment with SMM on my old hardware.

Posted: **Fri Sep 06, 2013 1:10 am**

i searched for a good way to do this for long time, the method i used before was switching back to full real mode, and restoring the idt to the real mode one , the ivt at address 0x0 , and i already struggled a bit with to mix protected mode 16 bit and 32 bit code , and then switching to real mode from a 16 bit code segment, as the cpu seem to execute certain opcode differently depending on the type of code segment (16 or 32 bits) even in protected mode, specially all the call/int/ret/iret and other opcodes dealing with the stack and calling convention, and some opcode seem to have a different behavior in 16 or 32 bit protected mode

it was already working, and could even be mixed with other code, but call would be very slow and would require to disable the interupt handling from protected mode, or would need to have handler that are registered in the ivt and could also work in real mode, but it was mostly used during first initialized stage, some of if could be done before the switch to protected mode, but i still prefered to keep a way to be able to call some bios function from protected mode even if it's only in very first stage where not much is loaded and no interupt are really enabled except the one that are already enabled in real mode (timer/keyboard etc)

but this way is better, and there is no code wrote in 16 bit segment at all , it make the call to 16 bit routines directly from 32 bit code segment, even if it need to push the good values in the good format in the stack manually and do the 'far jump' manually, it's a bit hacky but it shouldn't create too much problem, the format of the calling stack is rather standard and hardcoded in cpu anyway, not sure if there is a good and reliable way to be able to make a 16 bit far call from a 32 bit code segment, as assembler and compiler seem to be doing a bit what they want with how they create opcode for int/iret(d),pushf(d), pusha(a) and some far jmp synthax, even the way the cpu is supposed to interpet the opcode is not very clear to me yet, as the exact same opcode can have 16 or 32 bit version, depending on size prefix of the opcode i think, so it's a bit hard to know exactly how a particular instruction will be encoded, apparently gcc doesn't handle the 16 bits version of the instructions at all, and it assemble everything using db 66h prefix

there is still something that bios need to do when manipulating address that need to be carefull with, for example this entry

set_gdt_entry(new_gdt,&new_gdt_entries_ref,0x1490,0x00014900,0xFFFF,seg_access,flags);

segment selector => 0x1490
physical address => 0x00014900
address of the buffer => 0x000149XX

could be figured out, as i use a fixed address in the low memory as exchange area between 16 bit routines and 32 bits ones, but the way real mode addressing work make in sort that several combination of seg/ofset can point to the same address

in 'vanilla' real mode, normally i would expect segment register are used only to point on segment boundaries and address in them being accessed by the ofset pointer, so i compute the seg/ofset address with the segment pointing at 0x10000 (where the kernel is loaded), and then compute the ofset from the fixed address of the binary image

but when the address is a buffer, it seem bios prefer to have the segment register pointing at an address as close as possible of the start of the buffer, basically on the previous 16 byte boundary as the address in real mode segment register increment by 16 bytes boundaries, and then use the ofset starting from there, to be able to access the maximum memory of the buffer in a loop without needing to switch segment register, so real mode addresses for buffers should be computed with segment on the 16 byte boundary before the buffer, and ofset register with the ofset to the start of the buffer, like that it's more unlikely the bios will have to update the segment register value passed in, unless your request an operation on a buffer > 64k

there is only the issue with internal buffer, like in the atapi bios they seem to create a command on the stack, and then compute the real mode address with a segment register pointing at the beginning of this pointer on the stack, so sometime they might need to use segment register values pointing at internal buffer with arbitrary value, something should probably be done to add them automatically in the gdt if a page fault occur with that kind of error in code segment belonging to the bios in reserved memory area, and segment value that aren't multiple of 8 can be catched in the exception handler and << 4 to make a valid segment selector, it should work as long as it doesn't use this segment value to compute an address to make up another segment/ofset pair, or that it doesn't do weird thing, or that there is no conflict between sellector that are <<4 of the normal real segment mode value, and address that would use a real mode segment register of the same value, but it would be bad luck =) maybe something more tricky can be used to make up valid segment selector with a linear mapping to the original real mode segment value that couldn't be mistaken with another 'true' real mode address

like for example, let say the stack is set at physical address 0x1000, and is 0x1000 is size, and the bios need to setup a point to an address in it , like 0x1563 to make a pointer to the stack

it would probably make real mode mode address with a segment register value of 0x156 and ofset of 3, and 0x156 would not be a valid segment selector, so need to shift the value of the real mode segment by 4 (or at least 3) to be sure to get a valid segment selector, the segment selector would then become 0x1560, the memory access using this segment register would be correct from protected mode, but the value doesn't match the physical address it would have in real mode, and if by any bad luck, the bios then need to access a memory at 0x15600 , it could use a segment register of the same value to represent the address, and they would be undistinguishable of the segment selector pointing at address 0x1560

i need to look into vm86 mode it can be usefull for some things, but as far as i understand for now, it's not very suitable to run some routine and get back return result easily, it's more made to run independant applications in them, but not to call real mode routines with shared memory area or return result from the main program, like some function that could be used to setup some basic information about the system from the bios in the system memory, or to make fast call to real mode routine and get the result back

maybe some hook could be made also in hardware interupt handler to call the vectors in the ivt with the same method, in case the real mode code want to install some hardware interupt handler in the ivt to handle some interupt driven hardware

Posted: **Fri Sep 06, 2013 2:36 am**

h0bby1 wrote:i searched for a good way to do this for long time

Basically, the 8 bit instructions will run the same on both RM16 and PM32.
However, one of the fundamental differences between RM16 and PM32 is word size.

Code: Select all

.code16
  mov|add|sub|and|or|xor|etc 16_bit_reg, 16_bit_immediate

The same binary code when run in PM32 is equivalent to:

Code: Select all

.code32
  mov|add|sub|and|or|xor|etc 32_bit_reg, 32_bit_immediate

This effectively eats one, two or part of the next opcode(s).
In aeBIOS I use this principle to have the same binary code do different things depending on the mode in which it is called. From
RM16 to initialize aeBIOS and return in PM32 a far call is made to 0:0x2000. Optionally, (although typically not necessary)
a permanent return to RM16 from PM32 can be made from low memory by calling the same entry at 0x2000.

Code: Select all

.code16
.globl _start
_start:
  cmp ax, 0x1234
  jmp to_pm
.code32
  jmp to_rm

The cmp ax is basically a nop (although it does insignificantly modify the flags).The same binary code when run in PM32 is equivalent to:

Code: Select all

.code32
.globl _start
_start:
  cmp eax, 0x5BEB1234
  jmp to_rm

Posted: **Fri Sep 06, 2013 2:53 am**

the more tricky thing is that you can still have the 32 bit version of the opcode in a 16bit code segment by using opcode prefix, with bios it's not too much of an issue because it's called with the interupt mechanism, and bios are supposed to be as much cpu independant as possible, so use a well defined version of the opcode for iret

and yes also the way the cpu interpret some opcode that use a word size immediate, like the opcode 0xA3 , like mov ax,0xXXX, or mov ax,[0x00], will be encoded with something like mov imeditate value to register index 0, register index 0 can be ax or eax depending on the mode, and the size of the immediate value seem to be dependent on the 16/32, even if it's assembled with the exact same opcode not depending on the [bits] ('.code32') directive of the assembler in the way it is assembled, but on the 16/32 bit attribute of the code segment in which it is loaded, and you can also force the operation to be 32 bits in 16 bits mode with the prefix, but the opposite seems more tricky

from the infos on this page

http://ref.x86asm.net/coder32.html#modrm_byte_32

it seem the way the cpu interpret the opcode mod/rm bits depend on if the code segment is 16 or 32 bit, even real mode code can be run in a 32 bit code segment theorically and use 32 bits version of the instruction with the db 66h prefix

like if you switch back to real mode from a 32 bit code segment, i'm not sure if what predominate is the cpu flag that is in real mode, or the type of opcode 32/16 bit, but a switch the real mode from a 32 bit code segment would be most likely to make problem, even if the segment selector are mannaged properly, the way opcode are interpreted may change, i guess in real mode everything is assumed to use the 16 bit version unless the size prefix specify otherwise, while it may be different in protected mode

if i just change the code segment used to make the far call to bios function to 32 bit, it make some page fault in iret instruction because the value in the stack doesn't seem to match what the iret expect from a 32 bit code segment

this page also http://geezer.osdevbrasil.net/osd/intr/index.htm had infos on the difference between real mode /32 bit pm, but they don't speak about 16 bit real mode

theorically even 0x286 had a multi tasking system in pure 16 bit mode, so a pure 16 bit multi tasking/tss could even be expected theorically, but i don't think it's very likely to be found in any kind of software nowdays =) not sure if real mode even had a multi tasking system, i guess multi tasking require protected mode anyway

but it seem the behavior of exception/software interupt calling also differ between all the different mode

Posted: **Fri Sep 06, 2013 3:15 am**

h0bby1 wrote:the more tricky thing is that you can still have the 32 bit version of the opcode in a 16bit code segment by using opcode prefix

Bonjour h0bby1,

Yes, and in the new Compatibility Support Module BIOS there is a lot of 32 bit RM code which in PM32 will run as 16 bit with detrimental effect.
How do you plan on capturing the override prefix? And more importantly, how do you plan on resolving an override prefix where necessary?

Posted: **Fri Sep 06, 2013 3:19 am**

mikegonta wrote:
h0bby1 wrote:the more tricky thing is that you can still have the 32 bit version of the opcode in a 16bit code segment by using opcode prefix
Bonjour h0bby1,

Yes, and in the new Compatibility Support Module BIOS there is a lot of 32 bit RM code which in PM32 will run as 16 bit with detrimental effect.
How do you plan on capturing the override prefix? And more importantly, how do you plan on resolving an override prefix where necessary?

for now the only instruction that need handling is 0x8E, and target register are segment register, so only 16 bits in anycase, so i don't have to handle this for the moment, and this opcode can only have a register as a source as well, no immediate, so it's just 16 bit register as operand anycase

in other case, some more parsing would need to be done i guess, but normally it shouldn't create problem, all functions that are real mode are executed with a far call using a 16 bit code segment selector, and the only thing i need to be concerned about are segment register values, the rest i don't care if it's made in 32 or 16 bit, any of them should work the same

i guess it could create bugs if value would be passed as 32 bit register with unclean values in the upper 16 bits, but any real mode program that use 32 bit registers should first set their upper 16 bits to known value

but theorically, would just need to check if the byte fetched as the opcode in the above code is a valid prefix, and then fetching the opcode from the next byte and dealing with the mod/rm bits and the value accordingly to deal with some faulty code in an exception, but i don't see what could create an execption using the size over ride, if the program mannage it's data and stack as if it was in 32 bit , it shouldn't make any fault normally, it's only when memory access are done using wrong segment selectors, and they are always 16 from register so it's pretty simple

even if it's run in a 16 bit segment, it would execute them as 32 bit opcode, the 32 bit opcode is actually exactly the opcode with a prefix, so pm mode know how to handle those prefixed opcode automatically

you can perfectly use some portion of code assembled as 16 bit with the assembler directive, using prefixed opcode and making far call or far jump to it from a 32 bit segment, interupt are always far call and always push the segment register into the stack, sometime as 32 bit value sometime as 16, but the segment value is always 16 bits anyway, it's just stored in a 32 bit stack slot, but it doesn't create any problem normally otherwise

normally opcode should be almost identical from real mode to 16 bit pm, maybe there are some exceptions to this, but in most case it should be ok

Posted: **Fri Sep 06, 2013 4:12 am**

h0bby1 wrote:the rest i don't care if it's made in 32 or 16 bit, any of them should work the same

The point is, that they don't work the same. The override prefix makes a 32 bit instruction out of a 16 bit opcode in RM16 and
a 16 bit instruction out of the same opcode (which is 32 bit) in PM32.
There is also the issue of the near jmp, which has a 16 bit offset in RM16 and a 32 bit offset in PM32. In this case, when run
in PM32 the binary code will eat the next opcode as well.

normally opcode should be almost identical from real mode to 16 bit pm

PM16 is a totally different story.

OSDev.org

running 16bit real mode code in 32bit protected mode

running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode

Re: running 16bit real mode code in 32bit protected mode