running 32-bit code in LM64

[Octocontrabass]

You don't accept IBM mainframes as an example?

I don't. IBM mainframes have separate modes for 24-bit, 31-bit, and 64-bit software, the same way x86 has separate modes for 16-bit, 32-bit, and 64-bit software.

MIPS does not have modes.

[rdos]

You don't accept IBM mainframes as an example?

I don't. IBM mainframes have separate modes for 24-bit, 31-bit, and 64-bit software, the same way x86 has separate modes for 16-bit, 32-bit, and 64-bit software.

MIPS does not have modes.

Not really. 32-bit protected mode can run both real mode, 16-bit and 32-bit protected mode code. It was 64-bit mode that broke backwards compatibility by not extending protected mode with 64-bit registers, and dropping segmentation from long mode.

[Octocontrabass]

Not really. 32-bit protected mode can run both real mode, 16-bit and 32-bit protected mode code.

By switching modes. We're talking about binary compatibility without switching modes.

[nullplan]

Not really. 32-bit protected mode can run both real mode, 16-bit and 32-bit protected mode code. It was 64-bit mode that broke backwards compatibility by not extending protected mode with 64-bit registers, and dropping segmentation from long mode.

Compatibility mode, though. It did break compatibility to real mode by removing v8086 mode, but user space 16- and 32-bit protected mode software can still run. And the mode switch to compat mode looks exactly the same as the mode switch to 16-bit mode in protected mode, namely by loading the relevant segment into CS. Now, can you please stop arguing against long mode in bad faith? It is getting tiresome.

[kerravon]

You don't accept IBM mainframes as an example?

I don't. IBM mainframes have separate modes for 24-bit, 31-bit, and 64-bit software, the same way x86 has separate modes for 16-bit, 32-bit, and 64-bit software.

No - that's not correct. The mainframe AMODE is nothing remotely like the x86. Absolutely all instructions are available no matter what AMODE you choose. The AMODE just sets what MASKING their is on the address bits - because (what I consider to be) badly written software decided to use the upper bits of the address for storing some data.

AMODE 64 is really "don't mask", at least from the point of view of 32-bit software.

If IBM were to come up with a 128-bit machine then the only issue would be the same one the 32-bit systems see - negative indexes will need 2**64 to 2**65 mapped to 2**0 to 2**63.

And this model can go on for eternity.

And what's wrong with that model?

It's true that the IBM OSes don't do that negative index compensation - but the hardware itself is perfectly capable. And my own OS (z/PDOS) does that compensation.

Anything you wish to dispute? If not - why isn't the IBM/MIPS model the right way to do things for all processors from day 1? With the exception of if you use segmentation as the 8086 did, which is outside the flat model.

Segmentation is perfectly fine for the purpose it was designed to serve, but I do expect a recompilation when you switch from segmentation to flat.

[kerravon]

Not really. 32-bit protected mode can run both real mode, 16-bit and 32-bit protected mode code.

By switching modes. We're talking about binary compatibility without switching modes.

I think he might be right. PM32 allows you to define segments as being 16-bit. I think I asked this question before elsewhere but no-one knew the answer - there are some op codes that behave differently between RM16 and PM32. If those instructions are being executed in PM32 in a segment that has been marked (via the D-bit) as 16-bit, do the instructions get the RM16 behavior?

If not, then it would be a case of - RM16 code can run in PM32 so long as it doesn't use any reassigned op codes.

You would also need to map the selectors so that they are tiled appropriately. And keep data and code separate. And you may run out of selectors before you can map 640k - I've forgotten the numbers and a bing search didn't answer that immediately.

And you may not be able to directly manipulate storage outside of your own executable.

If you are writing pure C90 MSDOS applications (like the micro-emacs 3.6 you can find on UC8086 at http://pdos.org), it may work.

[Octocontrabass]

The mainframe AMODE is nothing remotely like the x86.

It's a mode selector that provides binary compatibility, exactly like x86.

The AMODE just sets what MASKING their is on the address bits - because (what I consider to be) badly written software decided to use the upper bits of the address for storing some data.

Software written to run on your OS can follow your rules. If you say all software that runs on your OS must not rely on AMODE to mask upper address bits, that's fine.

And what's wrong with that model?

Software written to run on a different OS (such as Windows) isn't going to follow your rules, so your Windows binary compatibility layer isn't actually compatible with Windows. What's the point of a binary compatibility layer when all software will need to be recompiled to work with it?

If not - why isn't the IBM/MIPS model the right way to do things for all processors from day 1?

Your question is invalid. IBM z/Architecture and MIPS do not follow the same model.

I think I asked this question before elsewhere but no-one knew the answer - there are some op codes that behave differently between RM16 and PM32. If those instructions are being executed in PM32 in a segment that has been marked (via the D-bit) as 16-bit, do the instructions get the RM16 behavior?

If the segment is marked as 16-bit, that's 16-bit protected mode. There are some instructions that behave differently between 16-bit real mode and 16-bit protected mode, but nothing that should affect an ordinary application.

If not, then it would be a case of - RM16 code can run in PM32 so long as it doesn't use any reassigned op codes.

Intel actually designed the 8086 with this forward compatibility in mind: applications were meant to treat segment registers as opaque tokens so that an unspecified future processor (which was eventually released as the 286) could change how segment registers were interpreted without changing application behavior. Any 8086 software that followed all of the rules will run perfectly fine in 16-bit protected mode.

Of course, the 8086 couldn't enforce good behavior without an external MMU, so there's a lot of software out there that completely disregards the rules, but the rules did exist for anyone who wanted to follow them. (I don't think any IBM-compatible PCs ever included an 8086 MMU.)

[rdos]

I think he might be right. PM32 allows you to define segments as being 16-bit. I think I asked this question before elsewhere but no-one knew the answer - there are some op codes that behave differently between RM16 and PM32. If those instructions are being executed in PM32 in a segment that has been marked (via the D-bit) as 16-bit, do the instructions get the RM16 behavior?

RM16 is handled with V8086 mode. RM16 use segment registers in other ways than protected mode (the base is the register value << 4). RM16 is entered by setting a flag bit. Protected mode have two different "operation modes" that depend on descriptors loaded in CS. One is PM16 where default addresses and operation size is 16-bit, and the other is PM32 where they are 32-bit. PM16 and RM16 (including real mode) allows the usage of 32-bit registers and operands by using the address size and operand size override op-code.

[devc1]

It's 2023 and you guys are still worried about 32 bit mode

Use 32 bit registers in 64 bit mode, it's as easy as it is.

Why do some people have to discuss a simple problems for years, pheww

[rdos]

Not really. 32-bit protected mode can run both real mode, 16-bit and 32-bit protected mode code. It was 64-bit mode that broke backwards compatibility by not extending protected mode with 64-bit registers, and dropping segmentation from long mode.

Compatibility mode, though. It did break compatibility to real mode by removing v8086 mode, but user space 16- and 32-bit protected mode software can still run. And the mode switch to compat mode looks exactly the same as the mode switch to 16-bit mode in protected mode, namely by loading the relevant segment into CS. Now, can you please stop arguing against long mode in bad faith? It is getting tiresome.

It's severely broken, mostly because compatibility mode thrashes upper 32-bits of registers. It's also broken because 64-bit registers cannot be used in protected mode, like 32-bit registers could be used in real mode and 16-bit protected mode. Also, long mode is essentially a "banked" 32-bit mode since 64-bit addresses cannot be used directly.

Also, it's possible to mix real mode, 16-bit protected mode and 32-bit protected mode seamlessly, which is not the case for compatibility mode that cannot handle 64-bit registers from long mode. AMD even claims that long mode cannot call compatibility mode, which turns out to be false though.

[rdos]

It's 2023 and you guys are still worried about 32 bit mode

Use 32 bit registers in 64 bit mode, it's as easy as it is.

Why do some people have to discuss a simple problems for years, pheww

Long mode lacks essential memory protection mechanisms, and so I will not write a 64-bit OS. You could use the upper 16-bits like a "selector", but popular C compilers cannot handle this properly either. Just like they never could handle segmentation properly.

[devc1]

What memory protection are you talking about, long mode has paging which includes PAT (on all modern processors) which includes Write protect, write combine, cache disable, write back, write through. What protection are u talking about, and what more do u need ?

U got large pages, huge pages. Multiple cpus, NUMA and SIMD. ......

[Octocontrabass]

It's severely broken, mostly because compatibility mode thrashes upper 32-bits of registers.

Other architectures work this way too. Why is it only a problem for x86?

It's also broken because 64-bit registers cannot be used in protected mode, like 32-bit registers could be used in real mode and 16-bit protected mode.

That's intentional. Allowing 16-bit software to use 32-bit registers is a major design flaw. A 16-bit OS won't save and restore 32-bit registers when it switches tasks, so all running tasks share the same set of registers!

Also, long mode is essentially a "banked" 32-bit mode since 64-bit addresses cannot be used directly.

You have to put the address in a register. Other architectures work this way too. Why is it only a problem for x86?

Long mode lacks essential memory protection mechanisms,

Most CPU architectures lack segmentation, so clearly it's not essential.

[kerravon]

It's 2023 and you guys are still worried about 32 bit mode

Use 32 bit registers in 64 bit mode, it's as easy as it is.

Code written with such 32-bit overrides will end up doing 16-bit if run on an 80386. Not what I wish to do.

[kerravon]

The mainframe AMODE is nothing remotely like the x86.

It's a mode selector that provides binary compatibility, exactly like x86.

I don't consider it to be anything remotely "exactly like the x86", where op codes do very different things depending on which mode you have selected. Simple address masking is a different concept.

The AMODE just sets what MASKING their is on the address bits - because (what I consider to be) badly written software decided to use the upper bits of the address for storing some data.

Software written to run on your OS can follow your rules. If you say all software that runs on your OS must not rely on AMODE to mask upper address bits, that's fine.

It's more of a hardware thing. Don't rely on address masking to exist. Software that follows the (conceptually simple) rule of "don't trash address bits expecting hardware to mask it for you" ALSO runs on standard IBM MVS (and z/OS), not just z/PDOS.

And what's wrong with that model?

Software written to run on a different OS (such as Windows) isn't going to follow your rules, so your Windows binary compatibility layer isn't actually compatible with Windows. What's the point of a binary compatibility layer when all software will need to be recompiled to work with it?

CERTAIN software - perhaps even yet to be written - is going to follow the rules. And if it doesn't, it will be considered a bug. If the vendor no longer exists and there is a bug, then I don't expect it to work.

I see a point in having rules. I will follow those rules myself. If no-one else follows them - so be it. But first I actually want some sensible rules to follow. That's the very first step. Which has taken me 37 years already. Once the rules are in place, then I will write and compile my code once.

Note that I don't really care that it has taken 37 years already. I posit that the gap between the 80386 and the x64 is 240 years, not the 24 years or whatever it actually was. I'm not trying to compete in some commercial market. I'm trying to create something that I (at least) consider to be "academically sound".

If not - why isn't the IBM/MIPS model the right way to do things for all processors from day 1?

Your question is invalid. IBM z/Architecture and MIPS do not follow the same model.

If you want to insist on your worldview, so be it - why isn't the MIPS (alone in the world supposedly) the right way to do things for all processors?

I think I asked this question before elsewhere but no-one knew the answer - there are some op codes that behave differently between RM16 and PM32. If those instructions are being executed in PM32 in a segment that has been marked (via the D-bit) as 16-bit, do the instructions get the RM16 behavior?

If the segment is marked as 16-bit, that's 16-bit protected mode. There are some instructions that behave differently between 16-bit real mode and 16-bit protected mode, but nothing that should affect an ordinary application.

Ok, cool, thanks. So it's just just the address default.

If not, then it would be a case of - RM16 code can run in PM32 so long as it doesn't use any reassigned op codes.

Intel actually designed the 8086 with this forward compatibility in mind: applications were meant to treat segment registers as opaque tokens so that an unspecified future processor (which was eventually released as the 286) could change how segment registers were interpreted without changing application behavior. Any 8086 software that followed all of the rules will run perfectly fine in 16-bit protected mode.

Of course, the 8086 couldn't enforce good behavior without an external MMU, so there's a lot of software out there that completely disregards the rules, but the rules did exist for anyone who wanted to follow them. (I don't think any IBM-compatible PCs ever included an 8086 MMU.)

That's fine - that's exactly what I want. I only care about well-behaved programs. And actually 8086 to 80286 is exactly one of the things I want to do - basically I want to create a PDOS/286 and I expect my RM16 programs to work.

This is another thing I don't quite have all the rules for. Specifically I think the MSDOS (actually PDOS/86) executables need to not have any functions that cross a 64k boundary. So that the PDOS/286 (also PDOS/386 running applications with the D bit set to activate PM16 for the applications so that more selectors are available) loader can load huge memory model programs and shuffle the data so that the exact same executable can address 512 MB of memory instead.

So Microemacs 3.6 built for MSDOS (read: PDOS/86) by tools that "follow the rules" can suddenly edit files up to 512 MiB in size instead of 640k.

As opposed to what happened in real timeline which is one day we woke up and that MSDOS executable said "sorry, this program cannot be run in 64-bit mode".

I don't want to go this V8086 route which continues the 640k limit. I want my executables to suddenly address massive amounts of storage - with an unchanged binary - and also unkludged (ie no conditional execution) binary.

I note that Microsoft/IBM had a "Family API". I don't know much about it, but I believe it contains some sort of stub to allow OS/2 1.0 emulation on DOS. I don't want a stub and I don't want conditional execution. I want exactly what you said above - follow Intel's rules and the definition of a selector suddenly changes, and your application doesn't care at all. Instead, your application says "whee - 16 MiB or 512 MiB instead of 640 KiB".

IBM and Microsoft probably have a business model that encourages the need for redevelopment for their new OS.

I instead have an academic interest in seeing if it is possible to follow Intel's rules (which I didn't know even existed until you told me just now). And I'm happy to write my own OS to make it happen. I realize I most likely lost that commercial competition 40 years ago - but I was never really in that commercial competition.