The RISC-V Privileged Architecture

Sv39, Sv48, Sv57 and satp, Measured on the Compiler's Own Shifts

27 minadvancedModule 1: The Address, the Bits, and the TrapContinues from rv-modes
27 min · Concept 2 of 4 · module: the address, the bits, and the trap · The RISC-V Privileged Architecture

Three formats, one number’s difference

RISC-V has three page-table formats and they differ by a single number: how many levels the walk has.

$ python3 rvpriv.py --run | sed -n '/^  format   LEVELS/,/^$/p'
  format   LEVELS   the leaf sizes below the root        and
  Sv39     3        4 KiB, 2 MiB, 1 GiB                  and one more, 512 GiB, at the root
  Sv48     4        4 KiB, 2 MiB, 1 GiB, 512 GiB         and one more, 256 TiB, at the root
  Sv57     5        4 KiB, 2 MiB, 1 GiB, 512 GiB, 256 TiB and one more, 128 PiB, at the root
                

That looks like a table to memorise and it is not, because each row falls out of arithmetic. Nine bits of the virtual address are consumed per level: 3 levels for Sv39, 4 for Sv48, 5 for Sv57. And the leaf sizes are not a separate fact — they are the same arithmetic read from the other end, because a leaf is any level whose PPN is aligned to its own size. A kernel that only uses 4 KiB pages has one leaf size available to it and is not making a choice; one that also maps 1 GiB regions is using the fact that the second level’s entries happen to be aligned.

The geometry is not a coincidence, and this is where a kernel usually goes wrong. 512 entries of 8 bytes is exactly one 4 KiB page. The specification says so directly: “Sv39 page tables contain 29 page table entries (PTEs), eight bytes each. A page table is exactly the size of a page and must always be aligned to a page boundary.” So the alignment requirement falls out of the geometry: a page table is a page, and a kernel that allocates one from a page allocator gets the alignment free.

That fact has a consequence two steps later that is easy to miss. Because a page table is a page, satp’s PPN is a page number rather than an address. Hold that thought — it is the reason the PPN field is 36 bits and not 44.

satp, and two masks of ours that are wrong on purpose

The layout is MODE at bits 63–60, ASID at 59–44, and the PPN below. On RV64 that leaves 44 bits, and “what is left” is the interesting arithmetic:

$ python3 rvpriv.py --run | grep -A 3 'is the interesting arithmetic'
  left" is the interesting arithmetic: 64 - 4 - 16 = 44 bits, and the PPN is
  stored DIVIDED BY THE GRANULE, so the register holds a page number rather
  than an address and the field is 44 - 8 = 36 bits of page number.
                

36 bits. And that is the point of the whole concept, because 36 is neither of the two numbers a reader reaches for. Forty-four falls out of the subtraction above, if you stop one step early. Fifty-four is the width of the PPN field in a PTE — a different register with a different job.

Both wrong answers are natural, and both compile without a diagnostic. This course does not merely tell you so; it ships both wrong masks in its corpus so that the wrongness can be measured:

$ python3 rvpriv.py --run | sed -n '/THE FIVE satp EXTRACTORS/,/^$/p' | tail -6
  field               slli    srli     asked for  emitted   bits it keeps    verdict
  MODE                (none)   0x3c     4          4         satp[63:60]      CORRECT
  ASID                0x4      0x30     16         16        satp[59:44]      CORRECT
  PPN                 0x14     0x1c     36         36        satp[43:8]       CORRECT
  PPN, 44-bit mask    0xc      0x14     44         44        satp[51:8]       WRONG by +8 bits
  PPN, 54-bit mask    0x2      0xa      54         54        satp[61:8]       WRONG by +18 bits
                

The widths and positions in that table are recovered from the shifts, not assumed. A slli by L ; srli by R pair on a 64-bit value returns input bits [R−L : 63−L], so the width is 64 − R and the low bit is R − L. Read the srli column and the answer is arithmetic: 0x3c gives 4, 0x30 gives 16, 0x1c gives 36.

Read the first row twice. The correct MODE mask emits no slli at all. The & 0xf is gone because it is provably redundant: a logical right shift by 60 of a 64-bit value leaves exactly four bits, so masking with 0xf cannot change it. And the ASID mask survives, because a right shift by 44 leaves twenty bits and the mask removes four of them. A confirmation that arrives as a missing instruction. If you wanted to know where the MODE field is, the compiler proved it for you — from the shift alone, without being told.

Now the consequence of the 54-bit row, which is the sentence this concept exists for:

$ python3 rvpriv.py --run | grep -A 3 'READ THE LAST ROW AGAINST'
  READ THE LAST ROW AGAINST THE THIRD. 54 bits of `satp` at [61:8]
  swallows the ENTIRE ASID, both bits of ASID's neighbours, and the low two
  bits of MODE. So `satp_ppn54_wrong` returns a value in which sixteen bits of
  address space identifier and two bits of translation mode are silently
  mixed into the page number.
                

Silently. A caller using that to compute a physical address gets a plausible number — small, aligned, in range — that is wrong in a way no assertion catches. And the 44-bit mask is worse in the way that matters, because 44 is the number the manual prints for a PTE’s PPN: the mistake looks documented.

Retraction R5 and R6 are both this, written down before they were corrected. The file’s own sentence for the lesson generalises past this architecture: two natural mistakes with a pleasing shape are worse than one ugly one, because the ugly one gets checked.

The PTE, measured as an instruction sequence

The manual draws a PTE with eight named bit positions. The artifact’s move is to write C expressions for those bits, compile them, and read the shift amounts out of the words — so the figure is checked by a compiler rather than by a second reading of the same figure.

$ python3 rvpriv.py --run | grep -A 4 'EIGHT BITS, EIGHT LEFT SHIFTS'
    EIGHT BITS, EIGHT LEFT SHIFTS, AND THEY ARE 0x3e 0x3d 0x3c 0x3b 0x3a
    0x39 0x38 in the order R W X U G A D -- that is, DECREASING BY ONE as the
    bit number increases, because 63 - n decreases as n does.
    And the right shift is 0x3f = 63 in all seven, because the
    mask is a sign-extend-to-64 and not a zero-extend.
                

63 − n is the only arithmetic in it. If the manual’s figure were wrong — if A were bit 5 and D were bit 6 — the compiler’s ((p) >> 6) & 1 would still emit slli by 0x39 and the check would fail. The compiler is not an authority on the PTE layout; it is an authority on what a C expression becomes, and those are two different things, and the composition of the two is what checks the figure.

One row of the eight is a different shape, and that is the trap. A reader looking for srli a0, a0, 6 ; andi a0, a0, 1 finds neither instruction: clang emitted a left shift and then a sign-extend, because that form is branch-free and constant-foldable. But bit 0 is the exception — for A and D alike, clang emitted andi a0, a0, 0x1 and no shift at all, because shifting by zero and masking with one is the identity and the optimiser knows it. So a helper written against the uniform shape works on seven of the eight rows and returns a wrong answer on the eighth with no diagnostic.

And the reserved bits are the most dangerous field in the format, because they are a trap rather than a no-op: bits 60–54 must be zeroed by software, and if any of them is set a page-fault exception is raised. The fault’s cause code says “load page fault” rather than “reserved bit set”, so the handler goes looking for a permissions problem and does not find one. The rule for a reader of this course: a PTE is not a struct with some padding. Every bit of it is either assigned or a trap.

What a page-table walk emits: three relocations, and an asymmetry

Here is where this course leaves the specification and goes back to bytes. A page-table walk on RISC-V is ordinary compiled code that happens to read a table, so it emits ordinary relocations — and reading them tells you things about the encoding that no diagram does.

$ python3 rvpriv.py --run | sed -n '/^  offset    code    name                  the symbol/,+8p'
  offset    code    name                  the symbol it names        addend
  0x0       23      R_RISCV_PCREL_HI20    root_pte                  +0
  0x4       24      R_RISCV_PCREL_LO12_I  .Lpcrel_hi0               +0
  0xe       23      R_RISCV_PCREL_HI20    l1_pte                    +0
  0x12      24      R_RISCV_PCREL_LO12_I  walk_three                +0
                

Read the symbol column. The HI20 records name the TARGET — root_pte, l1_pte, l2_pte, the three page tables, which is what you would expect. The LO12 records name something else entirely: they name the address of the AUIPC, and one of them names .Lpcrel_hi0, a label that is not in the source at all. The compiler created it so the ld four bytes later has something to point at.

That asymmetry is the load-bearing fact about a RISC-V relocation, and it is what makes the mechanism legible: a PC-relative address on RISC-V is a two-instruction arithmetic expression, and the object file stores one half of it in the instruction word and the other half in the relocation table.auipc puts the high 20 bits of (target − pc) in a U-type immediate; ld then has to add the low 12 bits, and the low 12 bits of a run-time address is not a compile-time constant. So the ld’s own immediate is left at zero and a record says “fill this in later”.

A reader who has only seen x86-64’s mov rax, [rip+X] will assume the RISC-V pair is a convenience and that the target’s address is somewhere in the record. It is not. That is why the corpus asks for %pcrel_lo(<the auipc’s own label>) rather than %pcrel_lo(target), and why getting it wrong is a linker error rather than a wrong number.

And the numbers in the words really are zero. Every immediate the LO12 records attach to is 0x000, read out of the object, cross-checked against llvm-objdump-21. That is the literal sense in which the address is half in the instruction and half in the table — and it is why a reader who hexdumps a RISC-V .o and computes an address from the instructions alone gets zero, with nothing in the file to say that the zero is a placeholder rather than a value.

The pairing rule, and the fact that nothing checks it here

The psABI requires each LO12 to be paired with a HI20 at a label it names, and the value that has to fit is the offset from that AUIPC to the target. A signed 12-bit field holds −2048 to 2047, so the AUIPC must be within 2048 bytes. And a 4 KiB granule is 4096, and 4096 > 2047 — which is where the familiar “the pair has to be in the same page” rule comes from, by arithmetic rather than by convention.

The artifact asks the assembler to violate it, six times, at six distances, and reads the records back:

$ python3 rvpriv.py --run | sed -n '/^  pad       HI20/,+6p'
  pad       HI20    LO12_I    HI20 at   farthest LO12 at  that distance  fits 12-bit?
  0x0       1       2         0x0       0x8               8              YES
  0x40      1       2         0x0       0x48              72             YES
  0x800     1       2         0x0       0x808             2056           NO
  0xffc     1       2         0x0       0x1004            4100           NO
  0x1000    1       2         0x0       0x1008            4104           NO
  0x2000    1       2         0x0       0x2008            8200           NO
                

Every one has one HI20 and two LO12s, including the case where the two LO12s are 8 KiB apart, and the assembler diagnosed none of them. Read the last column too: every immediate in every word is 0x000 at every distance, because the value is not knowable at assembly time and the assembler does not pretend otherwise.

What this course does not claim, and why that sentence is here. There is no RISC-V linker on this host, so whether that object is valid is a quoted claim and not a measured one, and the section says so rather than claiming a bug it cannot demonstrate. What it claims is narrower and worth more: the assembler produces the records and does not check them; the bound is arithmetic; and a linker is the only thing that can check it.

And the practical consequence for a kernel is the reason to care: the pairing rule is a constraint on code layout that the assembler will not tell you about, and a page-table walk is exactly the code most likely to violate it, because its references are spread across a data section and the author has no reason to think about where the AUIPC is. Real kernels keep the walk and its tables in one page, or accept the linker’s error. Neither can be tested here.

What was measured here, and what was not

This page has more measured bytes than any other in the course and not one page has been walked. The two facts are not in tension and it is worth being explicit about why.

Two findings on this page were retracted before they were published, which is the best kind. R9: the number of relocations a walk produces is not a property of the walk at all — it is a property of the data layout, and the same source at the same optimisation level emits one HI20 when three tables are adjacent and three when they are 4 KiB apart. The change is the merged-globals pass, and the evidence is in the symbol names. R13: the experiment that showed this holds the instruction count constant in the -fno-pic pair only; the default pair is confounded by a factor of four, and the section prints the confounding number in its own column rather than choosing the flattering pair.

Try it

  1. Compile the wrong masks and read the widths out of the machine code.(Open assets/samples/bits.c, find satp_ppn54_wrong, and compile it for riscv64-linux-gnu at -O2. Read the two immediate fields of the emitted slli and srli and compute the width as 64 − srli. You should get 54. Now ask what a 54-bit mask of satp returns when ASID is 0x1234 — and whether anything in the emitted code would tell a reader that answer was wrong.)
  2. Find the missing instruction.(The correct MODE extractor emits srli a0, a0, 0x3c and no mask. Work out, from the ISA alone, why the optimiser was entitled to delete the & 0xf — and then check that the ASID extractor, three lines away, was not entitled to. Two neighbouring expressions, same compiler, same optimisation level, opposite outcomes, and the reason is the width of the shift.)
  3. Break the pairing rule and find out who notices.(The corpus already does this at 0x2000. Read sp_0x2000.relocs in this directory: one R_RISCV_PCREL_HI20, two R_RISCV_PCREL_LO12_I, addends unchanged, and no diagnostic anywhere. Now write down what a linker would have to do to catch it, and mark that sentence QUOTED rather than measured. Being able to say which of your two sentences is a quotation is the actual skill.)
  4. Move a page table 4 KiB and count the relocations again.(In sp2.c, the three tables are declared adjacent or separated by 4 KiB depending on which build you ask for. Compile both ways at -fno-pic — the clean pair, because the default pair also unrolls the fill paths and confounds the count. Watch the HI20 count double with the instruction count essentially unchanged, and then look at the symbol names: .L_MergedGlobals becomes root, l1tab, l2tab. The author of the walk is not in the conversation. The compiler is.)

Connect

Back to concept 1, where the twelve-bit CSR address was decomposed and satp’s address appeared as one row among many — and where the accessibility and privilege bits explained why a user-mode program may not touch it. Forward to concept 3, which is what happens when the translation fails and something has to be told about it.

Out to mem-paging, which is the neutral course and owns the idea of a multi-level walk; nothing on this page re-teaches it. Out to a64-paging and x86-paging for the same subject on machines this collection can run — and read them for the contrast in method. Those two courses could observe a walk. This one counted the relocations it emits, which is a weaker fact about a different thing, and the weaker fact is the one that is checkable here.

And out to obj-relocations and reloc-types, which own what a relocation record is. The HI20/LO12 asymmetry on this page is a RISC-V fact and belongs here; the fact that the record has an offset, a type and a symbol is theirs.