The memory course measured the cost of a walk and this concept is the thing being walked. Four levels of nine-bit index, one of twelve bits of offset, and a 64-bit entry whose bits say whether the page is there, who may touch it, and whether the hardware has been here yet. The 64 bits divide into three groups and only one group is the hardware's.
And there is exactly one measurement in this course that observes a page-table entry directly. /proc/self/pagemap exposes one word per page of the calling process, and what that word contains is a single bit that moves: an untouched mmap region has no page-table entry at all, and the entry appears at the first touch — not at the mmap, not at the mprotect, at the touch. That is demand paging seen from the inside, through one kernel interface, and it is the closest a user process gets to watching the MMU work.
A 48-bit canonical address is cut into five fields and four of them are indexes:
$ ./sysdump | sed -n '/1B6\./,/^ PTE | 0 /p'
IDX | PML4 | 9 bits | address bits 47:39
IDX | PDPT | 9 bits | address bits 38:30
IDX | PD | 9 bits | address bits 29:21
IDX | PT | 9 bits | address bits 20:12
IDX | OFF | 12 bits | address bits 11:0, always added last
9+9+9+9+12 = 48, and with LA57 there are five levels of 9.
Four indexes of nine bits is 236 entries, and every level is itself a 4 KiB page holding 512 eight-byte entries, so a fully-populated 48-bit space would need 227 pages of tables and the whole walk would be four dependent memory accesses. That arithmetic is the reason the structure has four levels rather than one, and it is also the reason a TLB exists: a four-deep chain of dependent loads is a latency the hardware cannot afford on every access, so the top of it is cached and the bottom of it is the expensive part. The memory course's TLB concept owns that story.
Now the entry. Sixty-four bits, and the artifact divides them by who writes them:
$ ./sysdump | sed -n '/^ PTE | 0 /,/bits 9-11/p'
PTE | 0 | P | present; 0 makes the entry INVALID and every
PTE | | | access through it a #PF with error bit 0 clear
PTE | 1 | RW | writable; 0 with RW=1 above is the #PF case
PTE | 2 | US | accessible at CPL 3; 0 and CPL>0 is a #PF
PTE | 3 | PWT | write-through for this page
PTE | 4 | PCD | cache disabled for this page
PTE | 5 | A | accessed: SET BY HARDWARE on the first touch
PTE | 6 | D | dirty: SET BY HARDWARE on the first write
PTE | 7 | PS | page size: 0 = 4 KiB, 1 = 2 MiB, PD level ONLY
PTE | 8 | G | global: a translation usable by every address space
PTE | 9-11 | - | available to the operating system
And the rest of the entry, which is where the interesting flags hide:
$ ./sysdump | sed -n '/^ PTE | 12 /,/the entry is written by software/p'
PTE | 12 | PAT | at the PD level, the PAT bit for the whole 2 MiB
PTE | 13-58 | - | reserved, except bits 59-62 which hold PKRU
PTE | 59-62 | PKRU | protection key; ignored unless CR4.PKE or PKS
PTE | 63 | NX | no-execute; needs EFER.NXE, and a #PF without
PTE | | | the instruction bit set if the fetch is denied
PTE | 4-5 are the ONLY bits the HARDWARE sets. Everything else in
PTE | the entry is written by software, which is why a kernel's paging
PTE | format is a choice and not a fact.
PS at bit 7 is the one bit that changes the meaning of every other bit: set at the page-directory level, the entry stops being a pointer and becomes a description of a 2 MiB region, and bits 9 through 20 become physical address bits. One bit, two entirely different structures, in the same 64-bit word. And NX at bit 63 is the only protection bit that is on by default in a modern kernel, because it is the one that cannot be got wrong by a missing bit elsewhere: without EFER.NXE it is a reserved bit and every page is executable.
Six mmap regions, read out of /proc/self/pagemap at four points, and the only thing that changes is one bit:
$ ./sysdump | grep '^ PAGEMAP |' | head -6
PAGEMAP | a fresh RW mapping, untouched | 0x0080000000000000 | bit63 0
PAGEMAP | the same page after ONE write | 0x8180000000000000 | bit63 1
PAGEMAP | a PROT_NONE mapping, untouched | 0x0080000000000000 | bit63 0
PAGEMAP | PROT_NONE, then mprotect RW | 0x0080000000000000 | bit63 0
PAGEMAP | and then ONE write | 0x8180000000000000 | bit63 1
PAGEMAP | a PROT_READ page, after ONE read | 0x8080000000000000 | bit63 1
Four results in six rows, and each is a fact about the MMU rather than about a document.
mmap has no entry. Bit 63 reads zero, which in this interface means the page is not present — and the mapping is a PROT_READ|PROT_WRITE anonymous mapping that succeeded. The entry is created by the first fault, not by the mmap. This is demand paging, observed from the one side of it a user process can reach.mprotect alone changes nothing. The third and fourth rows are the same value: after mprotect from PROT_NONE to PROT_READ|PROT_WRITE the entry is still not present, and the fifth row is the first write. A permission change does not populate a page table, and the kernel does not have to do work to make a PROT_NONE region readable — the first touch is where the work happens.PROT_READ page is present after a read, and its word is not the same as the read-write page's: one bit differs and the artifact reports which. The artifact explicitly declines to name it, because bit 62 of a pagemap word is a mapping-sharing flag whose kernel definition has changed twice, and a page that names a bit it cannot account for is worse than one that admits it. That refusal is in the artifact's own prose and the harness asserts it.NX=1 for a page it had never written. The two bits share a number and nothing else.The page-fault error code is a fifth encoding with its own layout, and it is where the entry's bits become an event:
$ ./sysdump | sed -n '/The page-fault ERROR CODE/,/neither reaches/p'
ERR | 0 | P | 0 = the page was not present
ERR | 1 | W/R | 1 = the access was a WRITE, 0 = a read or a fetch
ERR | 2 | U/S | 0 = the access was at CPL 0, 1 = at CPL 3
ERR | 3 | RSVD| 1 = a reserved bit of the page-table entry was set
ERR | 4 | I/D | 1 = the access was an INSTRUCTION FETCH
ERR | 5 | PK | 1 = a protection-key violation
ERR | 6-31 | - | must be zero
Bits 1 and 4 are exactly the two a demand-paging system needs and
section 4 shows that ring 3 cannot see either of them.
And here is what ring 3 gets instead, measured three ways, and the third is the one that matters:
$ ./sysdump | grep '^ PFC |'
PFC | write to NULL si_code 1
PFC | write to a PROT_READ si_code 2
PFC | touch a PROT_NONE si_code 2
PFC | the first is a MAPPING failure and the other two are
PFC | PERMISSION failures, and the two permission failures
PFC | are the SAME number. The error code the CPU pushed has
PFC | a write bit and an instruction bit in it, bits 1 and 4,
PFC | and a demand-paging system needs both of them, and
PFC | neither reaches this process. si_code is the KERNEL's
PFC | translation of that word and it keeps exactly one bit:
PFC | present or not.
One bit of six. And the one it kept is bit 0, the present bit, which means the interesting measurement is not about a protection at all: it is that a user process can distinguish “this page is not there” from “this page is there and you may not do that.” That distinction is the whole of copy-on-write's user-visible half, and it is the only part of the error code that a process sees.
The two bits that are missing have a specific cost, and it is worth being concrete about it. Bit 1, the write bit, is what lets a kernel tell a COW read fault from a COW write fault and skip the copy in the first case — which is why a read of a shared page is fast and a write is not. Bit 4, the instruction bit, is what lets a kernel distinguish “the page is not executable” from “the page is not readable”, which is the entire difference between a PROT_NONE guard page and an ordinary unmapped hole. Both bits are pushed by the CPU, both are read by the kernel, and neither crosses into user space.
One row of the entry table is not measured here, and the reason is a failed experiment rather than a missing one:
$ ./sysdump | grep -A2 'THE PS BIT OF A REAL'
* THE PS BIT OF A REAL 2 MiB MAPPING. MAP_HUGETLB was tried and
returned ENOMEM on this machine, and no transparent huge page
collapsed during the run, so the PS=1 row of the entry table in
The PS row says 0 = 4 KiB, 1 = 2 MiB, PD level ONLY and this course has only ever exercised the zero. Every number on this page comes from a 4 KiB path, and the artifact says so in its limits block rather than leaving the reader to assume the huge-page path was measured too. A huge page changes the walk from four accesses to three and moves the RW, US and NX bits from the leaf entry to the directory entry, which is a different structure wearing the same word — and a reference that quoted a 2 MiB figure it had not measured would be quoting a story.
cd courses/x86sys/assets/samples && ./build_samples.sh, then read section 6D. (Expect bit 63 to be 0, 1, 0, 0, 1, 1 in that order, and expect the mprotect row to be identical to the row before it. On a machine with transparent huge pages the last row may differ, because a 2 MiB mapping collapses and the word changes shape — which is a result, not a failure, and worth chasing.)mprotect row. Add one read of the page between the mprotect and the pagemap read. (Expect bit 63 to become 1 there and the and then ONE write row to stop changing anything. The experiment is the same and the point is that a demand-paging kernel is lazy in a way that is observable, and the only reason most people never notice is that they never look.)A and D bits on a machine that will show you. This one needs a CPL 0 reader. (Expect A set by a read and D set by a write, and expect a kernel that does not clear them between processes to let a scanner find every page the process has touched — which is why every hardened kernel clears them and what the cost of clearing is. The bit that costs the most is the one that saves the most.)NX check and find out you cannot. Take a program that wants to know whether a page it owns is executable. (Expect that it cannot, from user mode, with any interface — and then expect that a page which is not executable is a fact it can establish by jumping to it and catching the fault. The asymmetry is the whole story: you can test a property by violating it, and you cannot test it by inspecting it. That is true of memory and it is true of most of what this course measures.)CR3 reload costs in each. (Expect five to be almost free if you have huge pages, because the walk is already dominated by the TLB miss. Then expect the CR3 sign-extension field to move from four bits to six and remember that a hypervisor presenting both depths to the same guest has to present two CR3 formats — which is a cost of the extension that appears in a place nobody looks for it.)Backwards, the memory course's address-translation concept measured what a walk costs and owns the TLB; its TLB concept is where the four dependent loads are cached. This page is the structure those loads are chasing, and the only thing it adds is the flag set and the one direct observation. The privilege course's harness concept is where the three protection failures were first measured as three signals; the second section here is the same three, with the error code that produced them printed beside them.
Forwards, and the connection is the tightest in the course. The previous concept's hole in the address space is why this structure has four levels rather than one, and the PS bit is why it sometimes has three. The next concept is about the thing that fills this structure when a fault arrives — and it establishes, by measurement, that the filling cannot be watched from user space even when the reading can.
Outward. The paging structure is the one place where x86-64's history is visible in the layout: a CR0.PSE bit that does nothing in long mode, a 4 KiB granularity that predates every other architecture's choice, and a global bit that exists because the original design had one address space. Every other architecture in this collection puts the same information in a different shape, and the comparison belongs to the memory course. What this course owes the table for is the bit positions, and the general form is worth keeping: four of the fourteen bits in a page-table entry are set by the hardware and ten by software, and the two the hardware sets are the two a kernel cannot predict.