Portability of page attributes / flags

[xenos]

In my kernel design, I have a separation between different levels of memory management: There is a physical page frame allocator ("chunker", because it manages chunks of memory, think of a buddy allocator, for example), a page table manager ("pager") and then there are "users" of these managers, such as the kernel heap, process / address space creation... In the last days, I was contemplating about the portability of page attributes / flags (access rights, caching policy, dirty / accessed), and the question: How should the pager (which manages virtual to physical memory mappings) communicate these flags to its "users" in such a way, that it can be ported between different architectures? For example, I had these thoughts:

• Expose the hardware flags to the "users" of the pager and let them pick the flags they want / need. Something like this:

  Code: Select all

  // User of the pager
  
  // INC_ARCH is a macro defined in config.h, which is generated at configure time, which the target architecture is chosen.
  #include INC_ARCH(Flags.h)
  
  Pager::Map(virt_addr, phys_addr, PAGE_USER || PAGE_RW);

  Code: Select all

  // arch/x86/ia32/Flags.h
  
  Enum PageFlags {
  // Flags such as PAGE_USER for IA32 architecture.
  }

• Define a set of abstract page flags (something like user readable / writeable / executable, supervisor readable / writeable / executable, write-back / write-through / uncached, dirty / accessed, global...), use them as parameters / return values of pager functions and let the pager translate between these abstract flags and whatever flags are supported by the underlying hardware. (IIRC, this is the way Linux handles it.)
• Define a set of specific, abstract memory types (user code / data, supervisor code / data, MMIO...), use them as parameters / return values of pager functions and let the pager choose appropriate flags.
• Let the pager provide specific functions to grant / revoke access rights, enable / disable caching, query dirty / accessed status...

So, out of curiosity, which way would you choose and why? Keep in mind that there are x86 and ARM page tables, MIPS, different page table layouts and the messy PPC paging mechanism, and the users of the pager should not be concerned with any of these. For example, the heap code invoking the pager should look exactly the same on these architectures, and the pager should do all the dirty work of dealing with different paging mechanisms.

[Brendan]

Hi,

So, out of curiosity, which way would you choose and why?

I would have "virtual types" that user-space uses; that have nothing to do with the CPU's types in the first place. For example, a page might be using the "read/write data type" (as far as user-space knows and cares), and that physical page might be mapped into the virtual address space as "read/write"; however it might also be "not present" because it's been sent to swap space, or "read-only" because it's full of zeros (and the page fault handler is doing an "allocate on write" thing), or "read-only" because it's part of a memory mapped file or the result of "fork()" (and page fault handler is doing a "copy on write" thing), or...


Cheers,

Brendan

[xenos]

I would have "virtual types" that user-space uses; that have nothing to do with the CPU's types in the first place.

I like this idea This not only abstracts from the underlying hardware, but also from the way the pager implements these page types.

[Antti]

Have you thought about adding the count parameter? At first glance, it looks like "Pager::Map" method will be called quite many times. Different page sizes might be another issue, e.g. what requirements will be set for virt_addr and phys_addr alignments? Would it make sense if you used page indices instead of addresses?

Code: Select all

Pager::Map(virt_page_index, phys_page_index, PAGE_USER | PAGE_RW);

There is a big disadvantage, however, because this assumes the page size is constant but perhaps this could be abstracted, e.g. always using the 4 KiB (or 8 KiB?) page size, even though the mapper could internally handle "4 KiB / 4 MiB" on x86 or "4 KiB/ 2 MiB" on x64 whenever it makes sense.

A lot of questions but no correct answers from me.

[xenos]

Actually in my code I do have parameters both for page size and number of pager to be mapped (Or rather in my design document, I haven't coded this part yet, at least not for the new version of my paging code.) I was just a bit lazy when making the forum post, so I used just the most simple call I could think of.

Another interesting thing to keep in mind is that ARM offers access level flags on a more fine-grained level than pages. You can have 4kB pages, where each 1kB subpage has different access rights.

[Antti]

As for the page size, it is interesting to note that UEFI embraces 4 KiB granularity in its interfaces. It does not matter after boot phases, of course, but this gives some long-term guidelines for what we could expect and rely on. What I mean is that perhaps we can rely on the assumption that at least the order of magnitude for a "granule" is something like "a few kilobytes" even in the distant future.

It is definitely interesting if an ARM-like, more fine-grained, approach becomes more common. Perhaps the only thing we can safely rely on is byte-granularity so maybe the virtual address space should be composed of "addr, size_in_bytes" areas and let the mapper handle granularities and roundings. In some (typical) cases it means "just less safe" but in some other cases it could mean "impossible".

[xenos]

Perhaps the only thing we can safely rely on is byte-granularity so maybe the virtual address space should be composed of "addr, size_in_bytes" areas and let the mapper handle granularities and roundings.

That's a good point, so the mapper should rather expose a function like this:

Code: Select all

bool Map(virt_addr, phys_addr, length_in_bytes, flags);

I also have some constexpr MinPageSize() and constexpr IsValidPageSize(log_2_number_of_bytes) exposed by the mapper, so that it's users can decide to partition their memory appropriately.

[Antti]

I am not sure whether the page size is the most on-topic detail but one note about it could be in order. We all know what happened with the disk sector size and how it effectively became a constant value. Although it is possible for users to partition their memory appropriately, it helps a lot if there really are some other sizes in use so code gets tested in practice. As a practical example (on x86 and x64), perhaps there could be a test build or a run-time flag that makes the OS use huge page sizes and everything should still work. It would be really inefficient, e.g. a lot of slack space, but the reason for that would simply be to prove robustness.

If you support many architectures from early on, this problem does not exist.