Back to course

The Encodings That Set Relocation Size

20 minadvancedModule 5: Emitting OneContinues from obj-emit
20 min · Module 5: Emitting One · Advanced

Why This Matters

Here is a relocation, and the question it does not answer:

  0000000000000004  R_X86_64_PC32  global_counter - 4

Everything up to this point has been about which four bytes, which symbol, and which arithmetic. Now ask the question a linker author actually has to answer: how many bytes is the field? The record does not say. The type implies it, sort of, and the only authoritative answer is the instruction encoding — which means a correct PC32 implementation requires knowing that mov with a RIP-relative operand is 7 bytes, that add is 6, that call is 5, and that the field sits at a different offset within each.

This is the concept where “every instruction, every architecture” stops being a slogan. The relocation table reduced 147 AArch64 types to seven flags. This is what those flags are applied to, and the reason a linker for a new architecture is a multi-month project rather than an afternoon.

And the failure mode is the sharpest in the course, because it is a truncation:

A 4-byte displacement cannot be patched into a 1-byte field. If an instruction encodes a displacement in 8 bits and the linker computes a value that needs 24, there is no way to write it. The linker must fail the link. A linker that truncates instead produces a binary that runs, jumps somewhere nearby, and misbehaves in ways that look like a logic bug rather than a toolchain bug. This is exactly what the TYPE flag exists to prevent — not to make the linker clever, but to make it refuse.

A Simple Model

Three questions, and they are asked in this order by any correct implementation:

Q1 WHERE IS THE FIELD? The record gives a byte offset. Converting it to a field requires decoding the instruction, because the offset is relative to the SECTION and the field is at a position that depends on the encoding. Q1a how many bytes is the instruction? Q1b at what offset within it does the field start? Q2 HOW MANY BYTES IS THE FIELD, AND IS IT SIGNED? From the encoding, not the record. A linker that assumes 4 because the type name ends in 32 is wrong on every architecture that has narrower forms -- and those are the ones that matter, because the narrow forms are what make short branches possible. Q3 DOES THE VALUE FIT? If not, the link fails. There is no third option.

And the reason Q1a and Q1b are separate questions is the whole concept in one line: the same 4-byte field type appears at different positions inside instructions of different lengths, so a linker that hardcodes an offset-from-instruction-start is wrong for every encoding that uses prefixes.

The clean case and the awkward case, side by side, on real bytes from enc_nopic.o (clang 21, -O0 -fno-pic):

  0x00:  8b 04 25 00 00 00 00        6 bytes, field at +3
          movl 0x0, %eax              R_X86_64_32S @ 0x03

  0x24:  48 8b 04 25 00 00 00 00     7 bytes, field at +4
          movq 0x0, %rax              R_X86_64_32S @ 0x28

  The two instructions are the SAME operation with a
  different register width.  The REX prefix makes the
  instruction one byte longer and pushes the field one
  byte later.  The relocation type is identical.

So the record says offset 3 in the first and offset 0x28 in the second, and in both cases the field is the last four bytes of the instruction. For these forms the rule “the field is the last four bytes” is a usable shortcut — and it is a shortcut that breaks the moment an instruction has an immediate after the displacement, which is the next entry in the table.

The Actual Detail

x86-64: the forms you will meet, with their field offsets

Every row measured from a real -fno-pic object in this course's assets/samples/:

  bytes              len  field at  instruction          reloc type
  ----------------   ---  ---------  ---------------------  --------------
  e8 00 00 00 00      5     +1      call rel32             PLT32
  e9 00 00 00 00      5     +1      jmp  rel32             PLT32
  0f 84 00 00 00 00   6     +2      jz   rel32             PC32
  8b 05 00 00 00 00   6     +2      movl (%rip), %eax      PC32
  8b 04 25 00 00 00 00 6    +3      movl 0x0, %eax         32S
  48 8b 05 00 00 00 00 7     +3      movq (%rip), %rax      PC32
  48 8b 04 25 00 00 00 00 7   +4      movq 0x0, %rax         32S
  c7 04 25 00 00 00 00 01 00 00 00
                      10     +3      movl $1, 0x0           32S

Four things fall out of that table, and each one is a rule a linker needs.

And the narrow forms, which is where truncation lives

The 4-byte field above is the common case, not the only one. x86-64 also has 8-bit and 16-bit displacement forms, and they are what make short branches and short jumps encodable at all:

  eb 00              2 bytes   jmp  rel8    field at +1, 1 byte
  70 00              2 bytes   jo   rel8    field at +1, 1 byte
  e8 00 00 00 00     5 bytes   call rel32   field at +1, 4 bytes
  0f 84 00 00 00 00  6 bytes   jz   rel32   field at +2, 4 bytes

  A rel8 reaches +/-128 bytes.  A rel32 reaches +/-2 GiB.
  The assembler picks the short form when the target is
  near -- and it can only know that after layout, which
  is why a linker may have to relax.

That last point is the one that matters for an emitter and it is easy to miss. If the assembler emitted eb (a 2-byte jump with an 8-bit field) and the linker later places the target 200 bytes away, the value does not fit in one byte. The 1-byte field holds 0xc8 = 200 as unsigned, or -56 as signed — and the jump silently goes to the wrong place. The relocation types R_X86_64_PC8 and R_X86_64_PC16 exist for exactly these, and the TYPE flag on them is what says: this field holds an address, so if the value does not fit, fail the link. A linker that truncates produces a jump to the wrong address with no diagnostic, and the resulting program is the kind of bug that gets reported as “my compiler is broken”.

AArch64: where the field is not even contiguous

Now the architecture that changes the shape of the problem, using real bytes from enc_a64_nopic.o. One reference to a global produces two relocations, and neither field is a little-endian integer sitting in the byte stream:

  0000: 90000008      adrp x8, #0
        relocation @ 0x00  R_AARCH64_ADR_PREL_PG_HI21   g32
  0004: b9400100      ldr  w0, [x8]
        relocation @ 0x04  R_AARCH64_LDST32_ABS_LO12_NC  g32
  0008: d65f03c0      ret

Decode the first word bit by bit. The file bytes are 08 00 00 90, so the little-endian word is 0x90000008:

ADRP encoding: 1 immlo(2) 10000 immhi(19) Rd(5) bit 31 op = 1 (ADRP, not ADR) bits 30:29 immlo = 00 2 of the 21 bits bits 28:24 fixed = 10000 (must be exactly this) bits 23:5 immhi = 00000 the other 19 bits bits 4:0 Rd = x8 the 21-bit SIGNED page delta = (immhi << 2) | immlo

So the 21-bit value the linker must write is split into 2 bits at the top of the word and 19 bits in the middle, with the opcode, a fixed marker, and the destination register interleaved. And the second instruction's field is a different bit range of a different word:

LDR (immediate) encoding: size(2) 111001 opc(2) imm12(12) Rn(5) Rt(5) bits 31:30 size = 10 (2 = 32-bit access) bits 29:24 fixed = 111001 bits 23:22 opc = 01 (LDR, 32-bit) bits 21:10 imm12 = the 12-bit page offset <- the second field bits 9:5 Rn = x8 bits 4:0 Rt = w0

Three consequences, and each one is a design requirement a linker must satisfy.

All three of those claims are checkable without an AArch64 machine, and aarch64_enc.py in the samples directory does the checking by building the words and asking a real disassembler to read them back:

  $ python3 aarch64_enc.py --build 3
  ADRP  word = 0xf0000008
        bits 30:29  immlo   = 3
        bits 23:5   immhi   = 0
        => 21-bit signed page delta = 3 pages = +12288 bytes

  and the same word, placed with .inst and disassembled:

  $ clang -target aarch64-linux-gnu -O0 -c adrp_test.c -o t.o
  $ llvm-objdump-21 -d t.o
  0000: f0000008   adrp x8, 0x3000        <- exactly +3 pages
  0004: f0ffffc8   adrp x8, 0xffffffffffffb000   <- -5 pages
  0020: 90000008   adrp x8, 0x0

That is a positive verification, not an absence of error. Encoding a known value and having an independent disassembler read back exactly that value is a much stronger claim than reading zeros and confirming they are zero — and it is the form of check the mission's rule actually asks for.

The forms that differ only in width

And the reason 44 x86-64 entries exist when the table above has eight rows. Every form has a narrower sibling:

  R_X86_64_8    1-byte absolute      R_X86_64_16   2-byte absolute
  R_X86_64_32   4-byte absolute      R_X86_64_32S  4-byte signed absolute
  R_X86_64_64   8-byte absolute      R_X86_64_PC8  1-byte PC-relative
  R_X86_64_PC16 2-byte PC-relative   R_X86_64_PC32 4-byte PC-relative
  R_X86_64_PC64 8-byte PC-relative

  and the general shape is a CROSS PRODUCT:
      ( absolute, PC-relative, GOT-relative )
    x ( 1, 2, 4, 8 bytes )
    x ( signed, unsigned )

So the count is not a list of 44 unrelated ideas; it is about nine behaviours expanded by a cross product of width and sign. That is the relocation table concept's claim, and this concept supplies the reason it is true: the width is a property of the instruction encoding, the sign is a property of whether the architecture treats the field as an address or as a number, and neither is a property of the idea being expressed.

A Real Example

Putting the two architectures side by side, and asking the question an emitter has to answer for each relocation it writes:

                      x86-64                    AArch64
  ------------------   ----------------------   ----------------------
  is the field         YES, usually.  Four        NO.  Bit ranges
  contiguous?          contiguous bytes, at      inside a 32-bit word,
                       a fixed position in       interleaved with the
                       the instruction           opcode and registers

  can you overwrite    yes, if you know the      NO, always.  Read the
  it?                  length                    word, mask, OR, write

  how wide can it       1, 2, 4 or 8 bytes       12, 16, 19+2, 21 or 26
  be?                   depending on the form     bits, none of them
                                                byte-aligned

  what does overflow   the TYPE flag: fail        the TYPE flag: fail.
  look like?           the link                  On ADRP the 21-bit
                                                field cannot hold a
                                                delta beyond +/-4 GiB,
                                                and the linker must
                                                refuse rather than wrap

  how many records     1                         2, for one reference
  per reference?

  what the compiler    the encoding              the encoding, and it is
  must do              and the layout            the layout -- which is
                                                why -fPIC changes
                                                instruction LENGTHS

The last row is the finding that ties this concept back to Position Independence, and it was measured rather than assumed. Switching to -fPIC on x86-64 changed the relocation offsets in the same file, from 0x04, 0x0a, 0x21, 0x33 to 0x05, 0x0e, 0x21, 0x33. The data references moved because REX_GOTPCRELX needs a REX prefix byte that the plain RIP-relative form does not, so the instructions grew by one byte and everything after them shifted. Changing the relocation type changed the instruction encoding, which changed every subsequent offset in the section.

That is a genuinely surprising coupling and it has a direct consequence for tooling: you cannot compute relocation offsets independently of relocation types, and a tool that walks instructions and records “there is a fixable field here” without knowing which form it is will be wrong on every -fPIC object. The offsets in a -fPIC object are not a shifted copy of the non-PIC ones; they are a different set.

Try It Yourself

$ cd courses/obj/assets/samples
$ llvm-objdump-21 -d -r enc_nopic.o
$ python3 aarch64_enc.py
$ python3 aarch64_enc.py --build -5

Check Your Understanding

Without looking back: a linker is applying R_AARCH64_ADR_PREL_PG_HI21 at offset 0 of a section, and the instruction there is the four bytes 08 00 00 90. The linker must write a page delta of 3. The file's little-endian word is 0x90000008. Where exactly do the three bits of the value 3 go, why can the linker not simply write four bytes, and what happens to the other 29 bits if it tries?

Apply It

You are writing the relocation-application pass of a linker for a new 32-bit architecture. The ISA has three addressing forms: a 4-byte absolute field, a 4-byte PC-relative field, and a 12-bit PC-relative field for short branches. The relocation table has eleven types. Your pass reads each record, computes the value, and writes it. It is correct on nine of the eleven types. The two it gets wrong are the 12-bit PC-relative ones, and the failure is that a program with a branch to a distant label links, runs, and takes the wrong path. What is wrong, what should the pass have done, and why is this the single most important behaviour to get right in a relocation pass?

The general habit: a narrowing write is the only silent error in a relocation pass. Range-check every field against its real width and signedness, and treat the check as mandatory rather than defensive.

Connect

This concept supplies the missing half of The Object Relocation Sets. That concept reduced 44 x86-64 types and 147 AArch64 types to seven property flags, and said a linker implements about nine cases. This is where the seventh flag, SIZE, and the eighth, TYPE, get their physical meaning.SIZE means “this field is a bit range, not a byte range, and it has a partner” — which is the AArch64 ADRP/LO12 pair. TYPE means “this field holds an address, so a value that does not fit is a link error, not a truncation” — which is the 12-bit branch. Two flags, two concepts, and this is the concept that explains what they are for.

The connection to Position Independence is the strongest in the course and it is measured. Switching to -fPIC changed the relocation offsets in the same source file, because the GOT-indirect form needs a REX prefix that the direct form does not, so the instructions grew and every subsequent offset moved. That is not a fact about PIC; it is a fact about encodings, and it is only visible once you have the field-offset table in front of you. The practical consequence is that relocation offsets and relocation types are not independent data — a tool that computes one without the other is wrong on every PIC object, and a backend that chooses a relocation type has implicitly chosen an instruction length.

The AArch64 read-modify-write requirement connects to the JVM's attribute mechanism and to a shared shape. Both formats have a field whose value is encoded across bits rather than bytes, and both were designed by people who had a reason for it. The JVM needed an attribute length that could grow; AArch64 needed an instruction set where every instruction is a fixed 32 bits and the immediate fields are as wide as the addressing modes require. Read-modify-write is the price of fixed-width instructions, and it is a price worth paying when the alternative is a variable-length encoding that would make every decode branchier. The mission's “read the waste to read the history” technique applies here directly: the interleaving of opcode and displacement is a design decision visible in the bit layout, and it is the same decision visible in an ARMv7 Thumb encoding's narrower fields.

There is also a direct connection to the WebAssembly course, and it is the sharpest cross-format lesson in the collection. WebAssembly's R_WASM_MEMORY_ADDR_LEB patches a LEB128 field whose width is variable — and it is the one relocation type in that format with no dedicated handling in wasm-ld's fast path. Compare the three cases: a fixed 4-byte field is trivial to write, a bit range needs read-modify-write, and a variable-width field needs the linker to choose a width and rewrite the instruction stream around it. Getting the width wrong in the first case is a wrong number; in the second, a wrong register; in the third, a misaligned instruction stream that desynchronises the decoder for every subsequent instruction. The severity escalates with how much the field's position depends on the value, and that escalation is the argument for treating width as a first-class property of a relocation rather than an implementation detail.

Finally, the emit concept's four bugs connect here, because one of them is this concept's subject. Bug 3 — the overlapping answer and answer_ptr — was found by checking a number, and bug 2 — the swapped sh_link and sh_info — by reading warnings. But a fifth bug that did not happen here, and would not have been caught by anything, is a relocation offset that pointed one byte off the field. On x86-64 that produces a binary that links and jumps to a slightly wrong address, because the field is a fixed four bytes and there is nothing to signal the mistake.The next concept is about how you catch a class of bug where every tool agrees with you and the answer is still wrong, and this is the canonical example of why an independent oracle has to be independent of the thing you are checking.

Last: the course is finished here. Proving Your Object File Is Right is the final concept, and it is the one that makes the other seventeen safe to trust.