Expand down stack segment setting on x86

[16bitPM]

Hi all,

I was tempted by this question of expand down segments, so I finally went ahead.
I used the well-known guides here and here.

I built up a real to protected mode program from scratch (see attachment) and then, when everything worked, I just switched the ED bit of my stack segment and converted the base and limits according to the guides. This is where things got funky.

First some code. My GDT looks like this:

Code: Select all

        ; GDT
GDT     dw SIZE_GDT
        dd  0  ; this is filled at later time
        dw 0
        descriptor <0ffffh,0,0,10011010b,0> ; code segment descr.
        descriptor <0ffffh,0,0,10010010b,0> ; data segment descr.
        descriptor <0ffffh-0400h,0,0,10010110b,0> ; stack segment descr.

Before going expand-down, the last entry was like this:

Code: Select all

        descriptor <0400h,0,0,10010010b,0> ; stack segment descr.

The critical section:

Code: Select all

        lgdt fword ptr [GDT]
        lidt fword ptr [NA]     ; load NULL-IDT (prepare triple fault)

        ;TEMP CHECK
        smsw ax
        or ax,1
        lmsw ax
        jmpf 8,@F
@@:
        mov ax,10h
        mov ds,ax
        mov es,ax
        mov ax,18h
        mov ss,ax                  ; <<< FAULTY INSTRUCTION
        add sp,0ffffh-400h      ; adjust SP

and probably most important of all:

Code: Select all

        mov bx,ss               ; Prepare SS descriptor
        xor ax,ax
        call realtolin
        add ax,400h             ; BASE = LA + LIMIT
        adc dl,0
	dec dl			; according to guide: - modulus!
        add di,8
        mov (descriptor ptr [di]).base_l,ax
        mov (descriptor ptr [di]).base_h,dl

So first of all: it worked perfectly with an expand-up segment, so all the other stuff seems to be working. I did various checks to confirm this.

Now, 2 things:

With the mov ss,ax instruction in place, the program is caught off via the "triple fault handler" (see full source code, I thought it's not really necessary to include it right here).
But when I replace this with mov es,ax, the computer hangs!
I also tried removing dec dl which seemed rather odd anyway (LA + Length - Modulus in Bob Smiths' guide) with nu apparent effect.
So I went ahead changed the code to this:

Code: Select all

        mov ax,10h
        mov ds,ax
        mov es,ax
        mov bx,18h

       lar cx,bx
       verr bx
       lahf
       mov al,ah
       verw bx
       lahf

and analyzed its results using debugx. As expected: correct AR byte, readable, writable.

Next stop: bochs. I figured I probably tripped an exception, so figuring out which one would definitely help. But this is completely weird now: the log is completely clogged with "BOUND_GdMa: fails bounds test". The clogging makes sense since the CPU places the address of the BOUND instruction on the stack, and the BIOS "handler" is just an iret. But why???
Interrupts are disabled, I do nothing with the stack, ... I really don't get it.

[Combuster]

But this is completely weird now: the log is completely clogged with "BOUND_GdMa: fails bounds test".

Which typically means you made a jump into garbage memory. And since it loops instead of triple-faults, I even expect it didn't even reach the (deprecated) smsw instruction, meaning the error is elsewhere.

The source doesn't contain the realtolin function btw.

[16bitPM]

Hi,

Thanks for answering.
First of all: my nickname may give some insight as why I use smsw
Second: on the native machine (AMD K6 running Windows 95 in DOS mode) the faulty program really DOES go through the triple fault.
Just to eliminate all possible doubt on this, 386SWAT (which catches RESET at F000:FFF0) confirmed this.

In the meantime I did some more tests. The program runs fine just before lidt [NA].
After that I can't do anything... yet!! I'm learning how to use 386SWAT for RM->PM transitions right now...

May I point out that just removing mov ss,ax (and change nothing else) works perfectly.
Extra test: loaded SS with the DS selector (10h). Now the program just falls through (no triple fault).
So... the error MUST be the load of the SS register, meaning either the selector or the descriptor is wrong. The selector value CAN'T be wrong, because it just works with selector 18h and an expand-up stack segment. So it must be the descriptor, but I kind of looked a zillion times at it.

Contents of GDT just before LIDT:

Code: Select all

1E1A:001A  20 00 BA E1 01 00 00 00    ; NULL-descriptor = GDTR
1E1A:0022  FF FF 30 E6 01 9A 00 00
1E1A:002A  FF FF A0 E1 01 92 00 00
1E1A:0032  FF FB 30 E6 01 96 00 00

So the limit is FBFF; later on SP gets adjusted to FFFF (which is correct) and the base is at the start of the code segment, growing down from 01E630 to 01E230.
I didn't deem my realtolin function worthy of posting. It's actually in another object file, but since you ask...:

Code: Select all

realtolin PROC
        mov dx,bx
        shl bx,4        ; preserve low order bits*16
        shr dx,0ch      ; dx=A16-A19
        add ax,bx       ; adjust offset
        adc dx,0        ; if HMA, flow over to A20
        ret
realtolin ENDP

But this is completely weird now: the log is completely clogged with "BOUND_GdMa: fails bounds test".

Which typically means you made a jump into garbage memory. And since it loops instead of triple-faults, I even expect it didn't even reach the (deprecated) smsw instruction, meaning the error is elsewhere.

The source doesn't contain the realtolin function btw.

[Combuster]

Well, since bochs is still the more verbose party, have you checked there aren't any messages printed before it goes looping on BOUNDs? If there aren't any, you might want to run the version with debugger enabled and just step through it to see where it branches off in the wrong direction.

[sandras]

I built up a real to protected mode program

my nickname may give some insight as why I use smsw

I understand it this way - you are building a real mode OS. So, if my understanding is correct, you're doing it wrong.

[16bitPM]

FOUND IT!!

There were no other messages in the bochs log. I have the version with debugger enabled but I don't know how to trap on the first instruction of my program (it's running inside a freedos image). But OK, compiled a new one with the magic BP (xchg bx,bx).
The program failed at int 21h. After examining the status of the segment registers at the point, I found this:

Code: Select all

ss:0x2fc8, dh=0x00009702, dl=0xfc80fbff, valid=1
	Data segment, base=0x0002fc80, limit=0x0000fbff, Read/Write, Expand-down, Accessed

The hidden descriptor was not updated!!! I guess I deserve some slapping

But thanks for the guidance.

[16bitPM]

I built up a real to protected mode program

my nickname may give some insight as why I use smsw

I understand it this way - you are building a real mode OS. So, if my understanding is correct, you're doing it wrong.

No, I have fun experimenting with 16-bit protected mode. I want the code to run on my old 286 too

[sandras]

I built up a real to protected mode program

my nickname may give some insight as why I use smsw

I understand it this way - you are building a real mode OS. So, if my understanding is correct, you're doing it wrong.

No, I have fun experimenting with 16-bit protected mode. I want the code to run on my old 286 too

I'm baffled and curious. I checked the wiki to refresh my knowledge about protected mode. I suppose you don't enable A20 line? Protected mode means 32-bit registers, so your "16-bit" must refer to the adress space size. Though in my eyes it's 21-bit, not 16-bit.

[Kazinsal]

I'm baffled and curious. I checked the wiki to refresh my knowledge about protected mode. I suppose you don't enable A20 line? Protected mode means 32-bit registers, so your "16-bit" must refer to the adress space size. Though in my eyes it's 21-bit, not 16-bit.

Code-ly speaking, it's 16-bit. However the A20 line does exist, and is enabled to allow segmented access to a 24-bit physical address space. You don't have 32-bit registers though, so you're still limited to 64K segments.

This is one of the reasons why it wasn't used a whole lot. It's quite a mess.

[16bitPM]

I'm baffled and curious. I checked the wiki to refresh my knowledge about protected mode. I suppose you don't enable A20 line? Protected mode means 32-bit registers, so your "16-bit" must refer to the adress space size. Though in my eyes it's 21-bit, not 16-bit.

1. No, I don't enable it explicitely at this time, but I don't go to extended memory so it doesn't matter anyway.
2. It has been shown that with HIMEM.SYS loaded and DOS=HIGH in config.sys, A20 actually never gets disabled. So again, my program is not affected by my sloppyness.
3. Protected mode does NOT mean 32-bit. Protected mode is there since the 80286 which is a 16-bit CPU. Actually, Windows 3.1 (which runs in protected mode) could also run on a 80286. In fact, the 80286 is just an 80186 with only protected mode extentions (even task switching features). The 80386 just extended the whole concept to 32-bit.
4. The 80286 has a 24-bit address bus.

So I can't even say I agree with the notion that it hasn't been used a lot. Windows 3.1 was the first windows OS which became hugely popular. Before that time though, I guess it was really rare except for some DOS-extenders in the late '80s.

[bluemoon]

Second: on the native machine (AMD K6 running Windows 95 in DOS mode) the faulty program really DOES go through the triple fault.
Just to eliminate all possible doubt on this, 386SWAT (which catches RESET at F000:FFF0) confirmed this.

I quick search I presume 386SWAT is a protected mode debugger, and you know what, triple fault with debugger has slightly different meaning as triple fault without debugger.

In the meantime I did some more tests. The program runs fine just before lidt [NA].
After that I can't do anything... yet!! I'm learning how to use 386SWAT for RM->PM transitions right now...

If I understand correctly, 386SWAT seems to be a DPMI/v86 host with features to interact with HIMEM.SYS and EMS manager; this means the machines has already put into protected mode, and you can't "switch to protected mode" twice.


Try again without any software debugger sitting behind.

[16bitPM]

I quick search I presume 386SWAT is a protected mode debugger, and you know what, triple fault with debugger has slightly different meaning as triple fault without debugger.

386SWAT lays dormant in the background but can be programmed to trigger on certain exceptions. A triple fault is sort of "catched" by a hardware BP at F000:FFF0. Only at this point will it enter PM.

If I understand correctly, 386SWAT seems to be a DPMI/v86 host with features to interact with HIMEM.SYS and EMS manager; this means the machines has already put into protected mode, and you can't "switch to protected mode" twice.

Not really, the technique is different (it intrudes into the PM context of a VCPI or DPMI host).
It is possible to debug raw RM → PM code if your code cooperates with 386SWAT, but the documentation is not really clear on how to achieve it.
Also see this text in the documentation:

Tips On Debugging RM Programs

Debugging Real Mode programs is quite similar to debugging VCPI programs, except that you must use the debugging interface to allow 386SWAT to gain a foothold into your PM context.; Because there is no interrupt (such as INT 67h for VCPI) when a RM programs enters PM, your program must cooperate with 386SWAT in order for it to be of use.

Also note that you cannot trace through the instructions which setup the resources the CPU needs to run in PM. In particular, this includes LGDT, LIDT, and MOV CR0,r32 (or LMSW), as well as those instructions which setup the segment registers in PM, especially CS and SS (note that CS is setup by executing a far jump).

Try again without any software debugger sitting behind.

Thanks but that was the situation to begin with. Anyway it's solved now (see posts above).

[16bitPM]

Which typically means you made a jump into garbage memory. And since it loops instead of triple-faults, I even expect it didn't even reach the (deprecated) smsw instruction, meaning the error is elsewhere.

I was also thinking: my findings mean that bochs doesn't really emulate a triple fault like it should be, at least in real mode.
On the real CPU, the attempt to call interrupt 21h immediately leads to #SS because the value of SP is invalid (400h < 0FBFFh), which immediately triggers a double fault (#SS handler cannot be called) which immediately leads to a triple fault (#DF handler cannot be called).

[rdos]

Which typically means you made a jump into garbage memory. And since it loops instead of triple-faults, I even expect it didn't even reach the (deprecated) smsw instruction, meaning the error is elsewhere.

I was also thinking: my findings mean that bochs doesn't really emulate a triple fault like it should be, at least in real mode.
On the real CPU, the attempt to call interrupt 21h immediately leads to #SS because the value of SP is invalid (400h < 0FBFFh), which immediately triggers a double fault (#SS handler cannot be called) which immediately leads to a triple fault (#DF handler cannot be called).

AFAIK, there are no limit checks in real mode, nor any double fault handler. The only fault that exists (except for single step) is invalid opcode. That means that if you have an "invalid stack" in real mode, the processor will just continue to execute garbage. If the garbage contains invalid instructions, you could end up executing the invalid instruction handler, but the consistency of the handler is not checked, rather it just jumps to any address coded in the IDT.

The same applies to V86 mode, except that some instructions are trapped to the V86 monitor (which executes in protected mode).

[Combuster]

AFAIK, there are no limit checks in real mode, nor any double fault handler. The only fault that exists (except for single step) is invalid opcode.

Limit checks are definitely enabled in real mode - otherwise there wouldn't been a difference with Unreal mode.

For that very same reason, GPFs work just fine, as do the majority of others: division by zero, debug, breakpoint, bounds failed, math fault etc.