Keyboard Shortcuts

Ctrl+K
Open search
Esc
Close search / dialog
↑
Back to top

The Hole Has No Fixed Address

28 minadvancedModule 3: What Mode 3 Is Made OfContinues from priv-convention
28 min · Exceptions, Privilege and Mode Changes

Why this matters

Ring 3 is usually taught as “less privileged,” which is vague. Here is what it actually is, and it is not a permission. A ring-3 process cannot name more than half the address space, and the reason is that the other half is not an address.

Every course in this collection that has mentioned canonical addressing has given you a one-line rule — “bits 63:48 must be all zero or all one” — and that rule is not wrong, but it is incomplete, and the incompleteness has a measurable consequence that will produce a wrong answer in a debugger, in a crash report, and in a fuzzing harness.

The consequence: a non-canonical access is not a page fault. It is not a permission failure. It is the CPU refusing to form an address at all. And when that happens, the address you would need to debug it is not delivered to anyone.

The model: one comparison, not a range

A 64-bit linear address is canonical when bit 47 equals bit 63. That is the complete rule for the 48-bit configuration, and it is one equality test:

canonical(a) == ((a >> 47) & 1) == ((a >> 63) & 1) 0x0000_0000_0000_1000 bit47=0 bit63=0 canonical 0x0000_3fff_ffff_ffff bit47=0 bit63=0 canonical 0x0000_7fff_ffff_ffff bit47=0 bit63=0 canonical <- top of low half 0x0000_8000_0000_0000 bit47=1 bit63=0 NOT <- 2^47 0xffff_7fff_ffff_ffff bit47=1 bit63=1 canonical 0xffff_8000_0000_0000 bit47=0 bit63=1 canonical <- bottom of high half 0xffff_ffff_ffff_ffff bit47=1 bit63=1 canonical The gap is [2^47, 2^64 - 2^47) -- 53% of the space, and none of it names memory.

And here is the trap in the familiar phrasing. “Bits 63:48 are all zero or all ones” is not the same rule. Test that phrasing against 0x0000800000000000: bits 63:48 are all zero, so the familiar phrasing calls it canonical, and it is not — bit 47 is set and bit 63 is clear. The correct statement of the rule is that bits above the implemented width must be a sign extension of the top implemented bit, and a parity test is how you say that. Every course in this collection that has written the shorter version has been wrong in a way that only shows up at exactly one address, which is the worst possible place to be wrong.

Two further wrinkles the manuals flag and short treatments omit. With five-level paging (CR4.LA57) the implemented width is 57 bits and the test is against bit 56. And CR3 has two Linear Address Masking bits, LAM_U48 and LAM_U57, which permit a user-mode address to have bits above the boundary non-zero. Both are real. Neither is on in the configuration this course measures, and the honest statement is that the 48-bit rule above is the baseline rather than the universal case.

The reality: the boundary is a function of the access

Probe it the obvious way — one 4-byte read, either side of 247 — and the boundary looks like it is at 247. It is not. Here is the same address probed three ways:

$ ./privbench | sed -n '/address  *1 byte/,/THE RULE/p'
   address                    1 byte   4 byte   8 byte
   2^47 -10 0x00007ffffffffff6       1        1        1
   2^47  -7 0x00007ffffffffff9       1        1      128
   2^47  -4 0x00007ffffffffffc       1        1      128
   2^47  -3 0x00007ffffffffffd       1      128      128
   2^47  -1 0x00007fffffffffff       1      128      128
   2^47  +0 0x0000800000000000     128      128      128
   2^47  +1 0x0000800000000001     128      128      128

   Bisecting separately for each width:
      1-byte  first rejected address 0x0000800000000000   2^47 minus it = 0
      4-byte  first rejected address 0x00007ffffffffffd   2^47 minus it = 3
      8-byte  first rejected address 0x00007ffffffffff9   2^47 minus it = 7

si_code 1 is SEGV_MAPERR: the CPU accepted the address and the page tables said no. si_code 128 is SI_KERNEL: the CPU rejected the address itself.

The rule, exactly:

first rejected address = 2^47 MINUS (access size MINUS 1) 1-byte -> 2^47 - 0 4-byte -> 2^47 - 3 8-byte -> 2^47 - 7

A read that starts below 247 still fails if it reaches 247. The CPU validates the whole byte range of the operand, not merely its first byte. That is why an 8-byte read dies seven bytes early and a 1-byte read gets right up to the line, and it is confirmed here three independent ways: by a twelve-row matrix and by a separate bisection per width, all agreeing.

Why this is the whole concept

Because it explains why a non-canonical access cannot produce a page fault. There is nothing for the page tables to look up, because there is no address. No present bit to clear, no protection key to check, no CR2 to load with anything meaningful, and no si_addr for the kernel to hand back. The kernel reports the fault with an address of zero:

What a handler actually receives: unmapped canonical address SIGSEGV si_code=1 si_addr = the address non-canonical address SIGSEGV si_code=128 si_addr = 0

Two faults. One signal number. The same si_addr a program is told is “the address that faulted.” And a fault handler that reads si_addr before checking si_code receives a null pointer and concludes the program dereferenced NULL — which is a completely different bug, in a completely different file, with a completely different fix. If you are writing a crash reporter, a fuzzer harness, or a debugger, this is the single most expensive distinction in this course.

There is a second-order version of the same trap, worth knowing because it will look like a contradiction. Intel's manual specifies that a non-canonical memory reference using RSP or RBP as the base raises #SS, not #GP, and that a segment override changes that back to #GP. So the vector a non-canonical access produces depends on which register formed the address. Nothing in the signal tells you which — the kernel has already collapsed it — and this course does not measure it, because from ring 3 the only observable is the same si_code 128 either way. It is listed in the artifact's limits rather than quietly omitted.

Worked: how the measurement nearly went wrong, three times

This result took three corrections to reach, and the corrections are the most useful part.

1. THE PREDICATE WAS WRONG. The first version tested "bits 63:48 all zero or all one", which calls 0x0000800000000000 canonical. The printed matrix therefore labelled the one address that mattered as fine. 2. THE BISECTION WAS BISECTING THE WRONG QUESTION. It searched for where the fault STOPS, on a machine where every candidate address faults. It converged on 0x0 / 0x1 -- a true statement about an unmapped page and a completely irrelevant one. It had to bisect on the si_code TRANSITION instead. 3. AND THE BISECTION THEN REPORTED A BOUNDARY INSIDE A SINGLE PAGE. 0x00007ffffffffffc / ...d. Impossible: nothing in x86 page granularity produces a 3-byte boundary. The cause was memset(&g, 0, sizeof g) on a struct that a signal handler writes. memset takes void*, so its writes are not volatile-qualified, the compiler elided them, and the record held values from a PREVIOUS probe. The lesson is wider than volatile: a measurement harness whose intermediate state is not volatile will hand you a plausible wrong answer rather than a warning. This one printed its result with total confidence and a two-row-tidy summary beside it.

What finally produced the matrix is not cleverer code. It is the observation that if the answer depends on the access, then ask with different accesses — which is the same move the memory course made when it stopped measuring one thing and started comparing two. A single probe gives one number; a probe repeated across a parameter gives a rule.

Try it. Add a 16-byte read to the three arms, rebuild, and check the prediction: the boundary should land at 247−15. If it does not, the course is wrong and you have found it in four lines.

Apply it

  1. Add the 2-byte and 16-byte arms and test the rule. Predicted boundaries: 247−1 and 247−15. (The harness group F asserts the existing three widths exactly, so a fourth width that breaks the rule will not be caught by it — you will have to extend the harness too, which is the point of exercise 2.)
  2. Extend the harness, then break the rule on purpose. Add the two widths to the artifact, rebuild, confirm the harness still passes, then change the access to a masked load that only touches one byte. Which group fails, and does the failure name the width or just say “the shape is wrong”? A check that cannot name what broke is a check you will spend an hour learning to trust.
  3. Write the crash-reporter rule. Your handler receives a siginfo_t. Write the exact sequence of tests that distinguishes: null dereference, unmapped read, write to a read-only page, and non-canonical access. (The answer: check si_code first, and treat 128 as “the CPU rejected the address; there is no address; do not trust si_addr”.) Then state what your reporter prints in each case.
  4. Work out how much of the address space ring 3 can reach, and check it against the page tables. With 48-bit canonical addressing and a 47-bit userspace limit, the practical figure is smaller than the architectural one. Read /proc/self/maps, find your highest mapping, and explain the gap between it and 247. Whose decision is each of the two boundaries, and which one does the CPU enforce?

Connect

Forward, the error-code concept takes the si_code values this one produces and asks what else the CPU told the kernel that a process cannot see. The three-architectures concept is where this stops being a hole: AArch64 uses the same implemented-bits-above-the-split idea, and the top half there is a device region rather than nothing, which is a different design answering the same question.

Backwards, the chain is direct and worth tracing. The convention concept established what happens when a ring-3 program asks for something; this one covers the space it asks from. Before that, the doors concept measured the cost of the crossing, and the two together are the complete story of a system call: here is what it costs, and here is the world you are allowed to name from where you land.

To earlier courses, this is the concept that gives the memory course's numbers their boundary. The translation concept measured what happens when a canonical address reaches the page tables with nothing there — a translation cost, a TLB cliff, a 3.29× ratio. All of that presumes the address is real. This concept is the layer above it, where the address is rejected before the page tables are ever consulted, and where “rejecting an address” turns out to be a surprisingly large part of what a privilege boundary does.

Outward, for anyone writing a kernel or a hypervisor, the practical consequence is short. If your fault handler reads CR2 and assumes it is a valid address, you will crash. And if you report the faulting address to userspace without checking whether the address was canonical, you will hand a user program a null pointer and a lie about what happened — which is a bug that will be reported as “my program dereferences null on some inputs” and will be unfixable from the other side of the boundary.

render_lesson_js(&mut page)