Keyboard Shortcuts

Ctrl+K
Open search
Esc
Close search / dialog
↑
Back to top

Three Error Codes and None of Them Visible

24 minadvancedModule 3: What Mode 3 Is Made OfContinues from priv-canonical
24 min · Exceptions, Privilege and Mode Changes

Why this matters

When an exception happens, the CPU pushes a 32-bit word on the stack that says why. It is the most information-dense thing in the whole mechanism: one word that distinguishes a not-present page from a protection violation, a read from a write, an instruction fetch from a data access, and a fault in user mode from a fault in the kernel.

Almost every description of system calls stops at the point where the word is on the stack. The interesting question is what happens to it afterwards — and the answer, measured below, is that a process never sees it at all. Not most of it. None of it. The kernel collapses eight bits into a signal number and one si_code, and what a userspace program can distinguish turns out to be much smaller than anyone would guess from reading the manual.

This matters for anything that debugs, fuzzes, sandboxes, or reports crashes across a boundary. And it has a second, structural point underneath it: x86-64 does not have one error-code format. It has three, and they are not related.

The model: three unrelated formats

The #PF format is a bitfield. It is not a selector, and reading it as one is the single most common error in fault-handler code.

#PF (page fault) -- a packed bitfield bit 0 P no translation: a present bit was 0 bit 1 W/R the access was a WRITE bit 2 U/S a USER-mode access caused it bit 3 RSVD a RESERVED BIT WAS SET IN A PAGING ENTRY -- not in the instruction bit 4 I/D the access was an INSTRUCTION FETCH bit 5 PK a protection key denied it bit 6 SS a shadow-stack access bit 7 HLAT a translation fault in the HLAT path bit 15 SGX an SGX access-control violation #GP / #NP / #SS -- looks like a selector bit 0 EXT the event came from outside the program bit 1 IDT the index names an IDT gate bit 2 TI ...in the LDT, not the GDT bit 3..15 the index itself bit 16..31 RESERVED, so the stack stays 16-aligned #CP (control protection) -- neither of the above bit 0..14 CPEC: 1 NEAR-RET, 2 FAR-RET/IRET, 3 ENDBRANCH, 4 RSTORSSP, 5 SETSSBSY bit 15 ENCL: it happened inside an enclave

Three shapes, one architecture, and the manual flags the divergence twice — once when describing the general format and once when describing #PF. There is no unified model because there is no unified mechanism: a selector error names a thing in a table, a page fault describes an access, and a control-protection error reports a class of instruction.

Bit 3 is the trap worth naming.RSVD does not mean the instruction had a reserved bit in its encoding, and it does not mean the access was of a reserved type. It means, verbatim, that no translation exists because a reserved bit was set in one of the paging-structure entries. It is a statement about a software error in the page tables, it pairs with bit 0 — bit 0 zero means not-present, bit 3 set means reserved-bit violation — and it is a completely different failure mode from every other bit, reached only by a kernel that has corrupted its own tables.

And the second thing #PF delivers is not an error code at all: it loads the linear address into CR2. The manual warns that a second page fault overwrites it, including a fault that happens while an earlier one is still being delivered. So a fault handler must save CR2 before doing anything that can fault again — which is why the double-fault machinery at vector 8 exists, and why the “use a separate stack via the IST field” mechanism is not optional.

The reality: what a process can actually tell apart

Six rows. One address arithmetic, three protections, three directions of access. Every row is a real fault on this machine, and every row's si_addr is exact to the page.

$ ./privbench | sed -n '/read  a PROT_READ page/,/exec  a PROT_NONE/p'
   read  a PROT_READ page         no fault
   write a PROT_READ page         si_code=2    SEGV_ACCERR  mapped, access not permitted  addr=0x511000
   exec  a PROT_READ page         si_code=2    SEGV_ACCERR  mapped, access not permitted  addr=0x512000
   read  a PROT_NONE page         si_code=2    SEGV_ACCERR  mapped, access not permitted  addr=0x513000
   write a PROT_NONE page         si_code=2    SEGV_ACCERR  mapped, access not permitted  addr=0x514000
   read  an UNMAPPED page         si_code=1    SEGV_MAPERR  nothing mapped here          addr=0x7bcd0e63f000

Two findings, and the second is the course's.

First: row 1 does not fault. PROT_READ means readable and the read succeeds. Rows 2 and 3 do fault — so a page that is readable is neither writable nor executable. W^X is enforced in hardware, from ring 3, and the address in the signal is exact. That is the good news, and it is worth having.

Second, and this is the one to sit with: rows 2, 3, 4 and 5 are four different events.

From ring 3, si_code cannot tell any of them apart. All four are 2. The one distinction it does make — 1 against 2 — is mapped, present but denied, against not mapped at all. Everything finer has been collapsed.

And those are exactly the two bits a demand-paging system needs. W/R is how the kernel decides whether a not-present page is mapped read-only or read-write, which is the entire mechanism of copy-on-write: a write fault on a page marked read-only means “this was a copy-on-write target” and the fix is to unmark it. I/D is how it decides whether to map the page executable. The kernel sees both. A process sees neither.

A 32-bit word, compressed to: one signal number (which of ~64 things) one si_code (1, 2, or 128) one address (or 0, meaning "there isn't one") That is the entire channel. The other five bits are for the kernel.

This is not a criticism of the design — the boundary is where the kernel is entitled to keep things to itself. It is a statement about what userspace diagnostics can conclude. A userspace crash reporter that says “the access was denied” is right. One that says “the program wrote to memory it should not have” is guessing, and its guess is wrong half the time.

Worked: two failures of the measurement, and what they teach about harnesses

Getting this table to mean anything took two corrections, and both are about the harness rather than the hardware.

1. SIX ROWS AGAINST ONE PAGE. The first version mapped a page, ran three faults against it, and unmapped it between rows. A fault longjmps straight out of the function, so the munmap never ran. The mapping leaked, and rows two through six measured a page in whatever state the previous row left it in. Every row now gets its own address AND reports whether its mmap succeeded. A fault-row harness that does not check the mmap prints six identical numbers and means nothing. 2. THE "UNMAPPED" ROW WAS NOT UNMAPPED. It used the constant 0x60000000 -- which is inside this process's own PIE image. So the row reported SEGV_ACCERR for an address assumed to be empty, and the table quietly lost its only SEGV_MAPERR entry. A second attempt parsed /proc/self/maps for a gap and found none, because a PIE process has almost no gaps below its image. The fix is neither a better constant nor a better scan: map a page, UNMAP it, and read that. The address that is certainly unmapped is one this process has just unmapped. And a third bug hid underneath: the row helper re-mapped the page before touching it, so of course the "unmapped" row was mapped. The helper needed a flag, not a better address.

The pattern across all three is the same, and it is the reason this course's harness concept exists. A measurement harness that cannot tell you it is broken will confidently print a table, and the table will have a shape. Row 1 was a legitimate “no fault” and row 6 was a legitimate “permission denied,” and the table between them was nonsense that looked entirely reasonable — six rows, six addresses, a monotonic-looking column.

So: after any run of this artifact, ask not only “did it produce output” but “does the output have the shape the claim requires.” Here that means the table must contain both an si_code of 1 and an si_code of 2. If every row says 2, one of them is wrong and the artifact cannot tell you which.

Apply it

  1. Decode a real error code. Find a way to see the raw 32-bit word. Under a hypervisor with a fault-injection tool, or in a kernel module, or with ptrace on a 32-bit child. Print it in binary and identify every bit. (Expect to find that P is 0 for a not-present page and 1 for a protection violation — the same fault, two different words, and confusing the two is how “my page is there but it says not present” happens.)
  2. Write the exhaustive userspace classifier. Given a siginfo_t, produce a table of every conclusion a program may draw. Include the non-canonical case from the previous concept. Then mark every row where the honest answer is “cannot tell” — and check that your list has at least one.
  3. Find the fourth error-code format. There are three in this course. Search the SDM for exceptions that push something and see whether a fourth shape exists. (There is at least one more, in the debug-registers and the virtualization paths, and the answer is that the architecture has no unified error model at all — which is the honest summary.)
  4. Explain why the reserved bits 31:16 of a #GP error code exist. Not “for alignment” — the manual's reason is worth reading exactly, and it is a stack-alignment requirement on a push, which means it is a performance decision encoded in an error format. What else in this course is a performance decision wearing a correctness costume?

Connect

Forward, the three-architectures concept answers a question this one raises: if the error code is the machine's only record, how do the other two record it? AArch64's answer is three CSRs, which is a richer interface than x86's single word — and it costs three register reads instead of one stack slot. RISC-V's answer is a cause enum plus an optional value, which is the closest of the three to what you would design if you were starting fresh.

Backwards, the previous concept produced two of the si_code values in this table and explained the third, and the reason a non-canonical access has no address is the reason this table's most useful column is not an address. The vector concept is where the “error code: yes/no” column of that table came from, and the rows marked “yes” are the ones with a format on display here.

Outward, the practical audience is anyone building a sandbox. A seccomp-BPF filter sees the syscall number, not the error code — it is a different channel, further out. A pointer-masking or shadow-stack system sees the #CP format, which is the third one here, and which exists because those features needed a fault class that neither the selector format nor the page-fault format could express. Each new hardware feature that needs to report its own failures has added a format, and the count has been three for twenty years, which suggests the design is stable and was never meant to be unified.

render_lesson_js(&mut page)