Does your user space processes know about physical memory?

[OSwhatever]

I'm trying to figure out if allowing user space applications have any knowledge about physical memory (pages). If you look at contemporary operating systems it seems like they have removed any physical memory knowledge and they only operate on virtual memory.

This has to do with shared memory, when a process gives another process pages. Should they then give them the pages mapped in a virtual area in the target process only, or give them the actual physical page address so that the target process can later map these pages if necessary. I've seen solutions where a process can give a copy a virtual memory area or parts of a virtual area instead so that is also possible solution. What do you think is a good way here?

Another simple question for your kernel, when you map physical regions like IO memory, does your user process get the physical address or some abstraction of it.

Main question: Is abstracting away physical memory knowledge in user processes a good thing or not?

[sortie]

Yes.

The whole idea of virtual memory is that physical frames are interchangeable and it doesn't matter which exact physical frame you are using. This concept falls a bit flat, perhaps, with micro-kernel drivers - but these things are kinda special objects in the first place.

[bluemoon]

You can't guarantee a piece of memory stay with same physic address after been swap in/out.
If you want to identify the memory, design some kind of handle instead.

[Combuster]

I'm trying to figure out if allowing user space applications have any knowledge about physical memory (pages). If you look at contemporary operating systems it seems like they have removed any physical memory knowledge and they only operate on virtual memory.

Most OSes certainly have handles for physical memory - and several of the more advanced bits of software actually do query for the actual amount of installed/available physical memory to establish the cache size they want to use without trashing and such.

[linguofreak]

Another simple question for your kernel, when you map physical regions like IO memory, does your user process get the physical address or some abstraction of it.

Generally, if a device has a fixed physical address, a userspace driver will make a request to the kernel that that address be mapped into its address space, possibly also specifying a virtual address that it should be mapped at (if the driver cares).

Main question: Is abstracting away physical memory knowledge in user processes a good thing or not?

For forward-compatibility, stability, and security, it's essential. It may be necessary to break this abstraction for userspace drivers, but only processes run as root (or otherwise given permission by the system administrator) should be able to get around it.

[OSwhatever]

Generally, if a device has a fixed physical address, a userspace driver will make a request to the kernel that that address be mapped into its address space, possibly also specifying a virtual address that it should be mapped at (if the driver cares).

I solved this by having named physical memory areas. The user process does not know the exact physical address and can only use an offset within this named area.

As physical addressees are abolished in user processes then the question is what kind of primitive should be used when sharing memory.

[azblue]

I had been wondering about something along these lines; rather than starting a whole new thread it makes more sense to just post about it here.

From the responses, it seems that the only interaction the OS ever lets a user-mode program have with physical memory is:
a) how much physical memory exists and/or is free
b) Mapping to physical memory used by devices

There's another scenario where I can see some control of physical memory being useful:

Let's say a program allocates m pages for an array. Later that array needs to be resized to m+n pages. One way to do that would be:
1. Find and mark as used m+n contiguous pages in that program's virtual space.
2. Copy m pages from the old array to the new array
3. Mark the old array as free

But step 2 would be more efficient if it could have the first m pages of the new array pointing to the physical memory of the old array; no actual copying would be needed then. Then the pages the old array pointed to would be marked as not present.

The OS would provide 2 functions for this:
MovePhysicalPages()
FreePhysicalPages()

The benefit is obvious, but something doesn't feel right about it; giving a user-mode program such low level access is rather unconventional. Should this be done, or not? Are there risks I'm not thinking about?

[Brendan]

Hi,

Let's say a program allocates m pages for an array. Later that array needs to be resized to m+n pages. One way to do that would be:
1. Find and mark as used m+n contiguous pages in that program's virtual space.
2. Copy m pages from the old array to the new array
3. Mark the old array as free

The benefit is obvious, but something doesn't feel right about it; giving a user-mode program such low level access is rather unconventional. Should this be done, or not? Are there risks I'm not thinking about?

While the pages need to be contiguous in the virtual address space; there's no need for the pages to be contiguous in the physical address space. This means that you'd just allocate N more pages (from anywhere in the physical address space) and map them (without copying the previously allocated pages).


Cheers,

Brendan

[BMW]

While the pages need to be contiguous in the virtual address space; there's no need for the pages to be contiguous in the physical address space. This means that you'd just allocate N more pages (from anywhere in the physical address space) and map them (without copying the previously allocated pages).

What if the virtual memory addresses directly after the array being expanded were used by, for example, another array? You would have to move the array to a new virtual address to keep contiguity wouldn't you?

The benefit is obvious, but something doesn't feel right about it; giving a user-mode program such low level access is rather unconventional. Should this be done, or not? Are there risks I'm not thinking about?

Maybe you could abstract it e.g. provide a ResizeAlloc() function, which checks if it can be resized without moving physical pages, if not it simply does it.

[rdos]

Simple answer: No. User space has no business messing with physical memory. That includes user space being denied the possibility to change attributes of linear memory pages as well.

[Brendan]

Hi,

While the pages need to be contiguous in the virtual address space; there's no need for the pages to be contiguous in the physical address space. This means that you'd just allocate N more pages (from anywhere in the physical address space) and map them (without copying the previously allocated pages).

What if the virtual memory addresses directly after the array being expanded were used by, for example, another array? You would have to move the array to a new virtual address to keep contiguity wouldn't you?

Yes; but that wouldn't be the kernel's virtual memory manager's problem.

Typically there's 3 layers. Physical memory management (which mostly just allocated and frees physical pages of RAM); virtual memory (which mostly just pretends to do what it's told), and something in the process that the process uses to control its virtual memory use (e.g. custom code, or a library that implements "malloc()" and "free()", or a garbage collector, or whatever).

The code in the process doesn't really know anything about what the virtual memory manager actually did. For example, the process might ask for 1234 KiB of RAM to be allocated at a virtual address and the virtual memory manager might only pretend to do this and actually allocate nothing (and wait until there's a page fault to actually allocate page/s); or the process might ask for a file to be mapped at a virtual address and the virtual memory manager might only pretend to load the file (and wait until there's a page fault to actually allocate page/s and load data from the file into those pages). In the same way, a process might actually be using real RAM but then the virtual memory manager might need that RAM and send the data to swap space. Note: this is what makes "virtual memory" virtual - it's just a big illusion where anything may or may not actually exist at any point in time. It explains why a process can't know about physical pages - the pages that a process sees are only an illusion, may not have any physical addresses at all, and if they do those physical addresses may change at any time.

In the same way, the kernel's virtual memory manager only really deals with pages. It doesn't know or care what the process is using any of those pages for. It certainly can't change the virtual address of something and then search through the process' data (and the stacks of all the process' threads) searching for pointers/references to anything in the area that it moved, because it only knows about "virtual pages" and doesn't know what is stored anywhere in any of those virtual pages.

Mostly; the virtual memory manager just provides functions to allocate "pretend memory", free "pretend memory", pretend that a file is at a virtual address, control access permissions (make page/s readable, writable and/or executable), etc. It could also provide a "move_pages()" function that shifts virtual pages to a different virtual address. However, it's the code in the process that decides what is where in its virtual address space (and uses the functions that the virtual memory manager provides to ask the virtual memory manager to do things). For example; the code in the process might ask the virtual memory manager to relocate something (and then fix up any pointers/references to the data that was shifted), and then ask the virtual memory manager to pretend that the (now empty) area is RAM again.


Cheers,

Brendan

[BMW]

Yes; but that wouldn't be the kernel's virtual memory manager's problem.

My point was that if the virtual memory after the array was used, the array would have to be copied to the new location when expanding (unless you moved physical pages in virtual memory). Sorry if it was unclear and I made you type up that epic explanation of memory management!