How to find the range of MMAPable virtual addresses in a program?

Engineering

Background information:

  1. I'm using 64 bit Arch.

  2. I'm writing an experimental custom allocator, so portable code is a stretch goal, not a requirement.

  3. I'm not using C or C++. I'm doing my syscalls directly.

According to this SO post: https://stackoverflow.com/questions/17671423/where-is-the-stack-memory-allocated-from-for-a-linux-process A program's virtual address space is organized like this:

------------------ <--- Top of the process address space
Stack (grows down)
v v v v v v v v v
------------------

(unmapped)

------------------ <--- Maximum stack size.


(unmapped)


-------------------
mmap
-------------------


(unmapped)


-------------------
^ ^ ^ ^ ^ ^ ^ ^ ^ ^
brk (grows up)
-------------------
BSS
-------------------
Data
-------------------
Text
-------------------

------------------- <--- Bottom or process address space.

Presuming this is correct, I'm trying to find the exact range of MMAPable address space here. Is it sufficient to find the lower bound by doing sbrk(0), page aligning up, and then ensuring that brk and sbrk are never called again? For the upper bound is it sufficient to assume 008FFFFF FFFFFFFF - SIZE_OF_STACK - A_GIGABYTE_OF_PADDING, and then page aligning down, is safe address space? I'd prefer a more exact and robust answer for the upper bound, but I'll accept a reasonable approximation that's guaranteed safe.

Posted May 30, 2021, 12:05 AM UTC

122
CodeLobe•

The linker specifies the memory map in the executable's file format: ELF / EXE.

Just because a range is available doesn't mean it's in the process page table. The OS keeps a page table and informs the CPU to use it to map between the logical (virtual) process address-space and physical address. memmap or virtualalloc on windows (IIRC?) tells the kernel to map more pages into the page table. Therefore, you should have the full 64bit range of (virtual) process address space available to use for the program. Whether you actually have enough physical memory to map all those pages is a different question, some of it can be paged in/out via disc storage to make it seem like you have more RAM (it's possible to allocate more than the physical RAM, but I advise against tempting the user's OOM killer). The page table (victual) memory map can be sparse, it doesn't have to be contiguous regions.

/proc/PID/pagemap https://www.kernel.org/doc/Documentation/vm/pagemap.txt

2
PL_DesignOP•

So far the solution I proposed in the question seems to be working.

1