Errors about long mode

[rdos]

Link: http://wiki.osdev.org/X86-64

For instance, this is wrong:
"Lastly it also important to note that any attempt to load a selector value into the FS or GS registers in long mode will automatically set their base addresses to zero, due to the previously mentioned semantics of long mode segmentation."

I've done a lot of debugging and single stepping in long mode by now, and this statement is totally wrong. The bases are not set to zero. It is the processor that ignores the base and limit. As soon as the processor is switched to compability-mode, the loaded base and limit is activated. In fact, segment register loads operates exactly the same way in long mode as in compability mode and protected mode, and also fire the same exceptions. This has some major implications for how CS and SS must be setup when switching between long mode and compability-mode. BTW, this applies to all segment registers, not just FS and GS.

This also strikes me as inaccurate:
"Linear addresses are extended to 64 bit (however, a given CPU may implement less than this)"

There is no way to extend long mode paging to 64 bit without breaking existing code. Therefore, long mode use 48 bit linear addresses, and any change to this would require a new paging mode (similar to the difference between protected mode paging and PAE paging).

[Griwes]

Linear addresses *are* 64-bit. They just have some limitations, like huge hole in the middle of address space. And 0xffffffff80000000 is surely 64-bit, even if long mode paging isn't able to cover full 64-bit address space.

Still, linear adresses use 64-bits, to check whether given address is canonical or not.

[Owen]

That quote from the Wiki is doubly wrong, as the base of the FS and GS segments is not ignored

SS does behave differently in long mode. In particular, the NULL selector may be loaded and used at CPL=0 and the processor will do this

[rdos]

Linear addresses *are* 64-bit. They just have some limitations, like huge hole in the middle of address space. And 0xffffffff80000000 is surely 64-bit, even if long mode paging isn't able to cover full 64-bit address space.

Still, linear adresses use 64-bits, to check whether given address is canonical or not.

No, 0xffffffff80000000 essentially is 0xffff80000000, because that is where it maps in the page tables. All the addresses in the middle are invalid and not possible to reference.

[rdos]

That quote from the Wiki is doubly wrong, as the base of the FS and GS segments is not ignored

SS does behave differently in long mode. In particular, the NULL selector may be loaded and used at CPL=0 and the processor will do this

It is also valid to dereference through default segment (DS or SS) even if these are loaded with NULL selectors. And I think it is valid to use FS and GS overrides even if these registers are loaded with NULL selectors. The whole segmentation logic is simply disabled, while segment register loads still operate the usual way. The only reason CS cannot be loaded with null is that it defines ring and bitness / operating mode. I keep DS, ES, FS and GS loaded with NULL (faster save/restore sequences), and SS loaded with a 4G flat selector, because otherwise I cannot easily switch between operating modes with far calls. Interestingly, far calls and rets seem to operate the usual way as well, which is required in order to be able to switch between long mode and compability mode.

While kernel SS must always be loaded with a zero-based selector in long mode (otherwise interrupts and exceptions will not work), CS can change freely. Having SS loaded with NULL selector causes protection -fault when switching to compability mode.

[Antti]

Linear addresses *are* 64-bit

No, 0xffffffff80000000 essentially is 0xffff80000000

I agree with Griwes viewpoint.

Code: Select all

mov rax, 0xFFFFFFFF80000000 ; As 64-bit as possible
mov edx, [rax]

mov rax, 0x0000FFFF80000000 ; As 64-bit as possible
mov edx, [rax]

Are those the same? I know that this is not what rdos meant but I think it is not reasonable to "add lower half and higher half" to be the virtual address space measured in bits.

[rdos]

Linear addresses *are* 64-bit

No, 0xffffffff80000000 essentially is 0xffff80000000

I agree with Griwes viewpoint.

Code: Select all

mov rax, 0xFFFFFFFF80000000 ; As 64-bit as possible
mov edx, [rax]

mov rax, 0x0000FFFF80000000 ; As 64-bit as possible
mov edx, [rax]

Are those the same?

No. The latter is invalid

I know that this is not what rdos meant but I think it is not reasonable to "add lower half and higher half" to be the virtual address space measured in bits.

What happens, apart from the check for valid addresses, is that the 16 high-order bits are truncated. That it is the higher addresses that are valid probably has to do with them being "negative". The processor can use RIP-relative addressing +/- 2G, which from address 0 means -2G to +2G.

[Combuster]

I see one more of rdos' self-contradictions. Let's all ignore this thread.

[linguofreak]

Link: http://wiki.osdev.org/X86-64

For instance, this is wrong:
"Lastly it also important to note that any attempt to load a selector value into the FS or GS registers in long mode will automatically set their base addresses to zero, due to the previously mentioned semantics of long mode segmentation."

I've done a lot of debugging and single stepping in long mode by now, and this statement is totally wrong.

What you find by debugging and single stepping may be behavior particular to the particular processor you're using, unless you can find something, unless you can find something in AMD's manuals that specifically says that a compatible x86-64 implementation will behave in exactly that way.

This also strikes me as inaccurate:
"Linear addresses are extended to 64 bit (however, a given CPU may implement less than this)"

There is no way to extend long mode paging to 64 bit without breaking existing code. Therefore, long mode use 48 bit linear addresses, and any change to this would require a new paging mode (similar to the difference between protected mode paging and PAE paging).

There is no way to extend LM paging to 64-bit without breaking existing *paging* code.

Application code, or system code that didn't touch paging, would be able to run on a processor that implemented a larger virtual address space, and to use pointers with as high a most-significant-bit as the processor supported, without modification or recompilation, so long as it didn't explicitly check pointers for 48-bit canonicality.

Thus, x86-64 virtual addresses are 64-bit, but current implementations only allow 48-bits worth of the address space to actually be mapped.

[rdos]

What you find by debugging and single stepping may be behavior particular to the particular processor you're using, unless you can find something, unless you can find something in AMD's manuals that specifically says that a compatible x86-64 implementation will behave in exactly that way.

That particular processor is a (real) dual core AMD processor. I've learned to always trust reality rather than documentation if and when they conflict.

I'm sure my dual core Intel Atom processor behaves exactly the same way.

BTW, here is a reality-check for those that don't believe this to be the case:
1. Assume compability-mode is executing.
2. A hardware IRQ occurs, which causes a task-switch to whatever code
3. The task switcher code saves the segment registers of compability-mode somewhere
4. The task switcher later wants to switch back to compability-mode
5. The segment registers (for compability-mode) are reloaded in long mode.
6. Long mode does an ireq back to compability-mode
7. The bases and limits for the segment registers now are used again (because compability-mode should look like protected mode)

If the segment registers loaded in step 5 were not loaded with real bases & limits, the correct base and limit would not be in the selector caches in step 7 because iret doesn't restore general segment registers (DS, ES, FS, GS). In fact, there is no way compability-mode could be supported by the processor if segment loads in long mode didn't operate the same way as in protected mode. The processor actually also loads CS and SS bases and limits as it does the iret (which also happens in long mode).

And contrary to the initial information from AMD, it is possible to do far calls and returns both from long mode to compability-mode and from compability mode to long mode.

[rdos]

I see one more of rdos' self-contradictions. Let's all ignore this thread.

Nice to see that you at least fixed the error in the wiki.

[linguofreak]

What you find by debugging and single stepping may be behavior particular to the particular processor you're using, unless you can find something, unless you can find something in AMD's manuals that specifically says that a compatible x86-64 implementation will behave in exactly that way.

That particular processor is a (real) dual core AMD processor. I've learned to always trust reality rather than documentation if and when they conflict.

When they do in fact conflict, that's a good thing. When the documentation specifies a certain characteristic, but not its implementation, however, it can get you into trouble if you rely on implementation-specific behavior.

That said, taking a look at AMD's documentation, specifically, section 4.5.3, it does seem to be at least strongly implied, though not stated right out, that segment register loads in long mode do load the exact contents of a DTE into the segment register in question, and that the base and limit are subsequently ignored.

[bluemoon]

For instance, this is wrong:
"Lastly it also important to note that any attempt to load a selector value into the FS or GS registers in long mode will automatically set their base addresses to zero, due to the previously mentioned semantics of long mode segmentation."

Yes, this is misleading, according to Intel manual Vol 3.4.4

In order to set up compatibility mode for an application, segment-load instructions (MOV to Sreg, POP Sreg) work normally in 64-bit mode.
...
Normal segment loads (MOV to Sreg and POP Sreg) into FS and GS load a standard 32-bit base value
in the hidden portion of the segment descriptor register. The base address bits above
the standard 32 bits are cleared to 0 to allow consistency for implementations that
use less than 64 bits.
...
The hidden descriptor register fields for FS.base and GS.base are physically mapped
to MSRs in order to load all address bits supported by a 64-bit implementation. Software
with CPL = 0 (privileged software) can load all supported linear-address bits
into FS.base or GS.base using WRMSR.

So yes, you can directly load FS.base and GS.base in order to affect compatibly mode - but the base are truncated into 32-bit.

Bottom line, in kernel you would do swapgs anyway to quickly reload it (I presume it do not recheck some things), so directly loading fs/gs is less useful in kernel.

[rdos]

For instance, this is wrong:
"Lastly it also important to note that any attempt to load a selector value into the FS or GS registers in long mode will automatically set their base addresses to zero, due to the previously mentioned semantics of long mode segmentation."

Yes, this is misleading, according to Intel manual Vol 3.4.4

In order to set up compatibility mode for an application, segment-load instructions (MOV to Sreg, POP Sreg) work normally in 64-bit mode.
...
Normal segment loads (MOV to Sreg and POP Sreg) into FS and GS load a standard 32-bit base value
in the hidden portion of the segment descriptor register. The base address bits above
the standard 32 bits are cleared to 0 to allow consistency for implementations that
use less than 64 bits.
...
The hidden descriptor register fields for FS.base and GS.base are physically mapped
to MSRs in order to load all address bits supported by a 64-bit implementation. Software
with CPL = 0 (privileged software) can load all supported linear-address bits
into FS.base or GS.base using WRMSR.

So yes, you can directly load FS.base and GS.base in order to affect compatibly mode - but the base are truncated into 32-bit.

Bottom line, in kernel you would do swapgs anyway to quickly reload it (I presume it do not recheck some things), so directly loading fs/gs is less useful in kernel.

Does that mean that loading FS and GS in long mode directly updates the base? Does it mean that if wrmsr is used to load FS.base or GS.base, it makes it possible for compability-mode to access memory above 4G through FS and GS? I think that might be worth testing.

[bluemoon]

I don't know, but I'm interested too and would you do the test?