Back to course

The Limits That Shaped the Table

20 minadvancedModule 1: The VocabularyContinues from reloc-why-so-many
20 min · Module 1: The Vocabulary · Advanced

Why This Matters

Here is a complete object file, six bytes of code, and it contains a number that is wrong:

$ cat far.c
extern int target;
int reach(void){ return target; }
$ clang -O1 -fno-pic -fno-pie -c far.c -o far.o
$ llvm-objdump-21 -d far.o
0000000000000000 <reach>:
       0: 8b 05 00 00 00 00    movl (%rip), %eax   # 0x6 <reach+0x6>
       6: c3                   retq

Read the bytes, not the annotation.8b 05 is the opcode. The four bytes after it are the displacement: 00 00 00 00. The disassembler helpfully annotates the resolved-looking address as 0x6, which is where the next instruction is — not where target is. target is not at offset 6 of this object file. It is not in this object file at all.

So the object file contains a placeholder, and this is the single most misread fact in the whole subject: the displacement in an object file is zero, and reading a disassembler’s resolved annotation as if it described the object file will send you looking for a symbol at the wrong place. The Object Files course called this “the hole” and measured it; what follows is why the hole is the right shape.

A Simple Model

Three limits, and each one is a fact about the instruction set rather than about the linker.

LIMIT 1 THE DISPLACEMENT IS 32 BITS, SIGNED reach = ±2 GB from the END of the instruction x86-64 has NO absolute 64-bit addressing mode. Every 64-bit address on this chip is reached either PC-relatively or through a pointer the program loads. That is not a compiler limitation. The ISA does not have the instruction. LIMIT 2 A 64-BIT VALUE DOES NOT ALWAYS FIT IN 32 BITS so there are two 4-byte absolute relocations: R_X86_64_32 truncate and carry on (WRONG ANSWER) R_X86_64_32S fail the link (RIGHT ANSWER) LIMIT 3 ONE INSTRUCTION, ONE REACH so a reference beyond ±2GB cannot be patched. It has to become MORE INSTRUCTIONS.

Limit 1 is the one that surprises people who have only used 32-bit x86. On a 64-bit machine you still cannot put a 64-bit address in an instruction. The address space is 64 bits wide and the widest thing you can name in one instruction is 32 bits of displacement. Everything else is arithmetic the program does at runtime.

Limit 2 is a policy encoded in a type name, and it is the only place in the vocabulary where the format is choosing between a wrong answer and no answer. Limit 3 is the one that changes the linker’s job from patching to generating.

Does It Actually Work?

The zero displacement, confirmed at the byte level, and the addend that fills it:

$ xxd -s 0 -l 8 far.o | tail -1
00000000: 8b05 0000 0000 c3                   .....
              ^^ ^^^^^^^^ ^^
              |  zero displacement
              opcode

And what the linker writes there once the address is known:

$ cat far2.c
extern int target;
int reach(void){ return target; }
int main(void){ return reach(); }
$ clang -O1 -fno-pic -fno-pie far2.c -o far2
$ objdump -d --start-address=0x401000 --stop-address=0x401010 far2 | tail -3
  401000: 8b 05 2a 10 40 00    mov  0x402a2a(,%rip),%eax
                                ^^^^^^^^^^^^^^
                     402a2a  <- the real target, filled in by the linker

Same opcode, different four bytes. The object file says “the displacement is up to you, linker”; the executable says 0x0040102a, which is 0x401006 + 0x1a24 — the distance from the end of the instruction to the symbol. That distance is what R_X86_64_PC32 computes, and it is why the addend exists at all: the field is written once, by the linker, and the arithmetic S - P is not something that could have been precomputed.

Now limit 2, and the two four-byte absolute forms side by side:

$ cat abs.c
extern char far_away[4096];
char *get(void){ return far_away; }
$ clang -O1 -fno-pic -fno-pie -c abs.c -o abs.o
$ llvm-objdump-21 -r abs.o | sed -n '/.text/,/^$/p'
0000000000000001 R_X86_64_32S  far_away

Clang chose 32S here, not 32. That is a choice with a reason: the address of a 4096-byte array in an executable can plausibly exceed 4GB on a system with a large mapping, and silently truncating it would produce a binary that runs and reads the wrong memory. The S makes the linker check instead.

And you can watch the check fire, which is the satisfying part:

$ cat addr.c
extern char huge[1];
unsigned long get(void){ return (unsigned long)&huge; }
$ clang -O1 -fno-pic -fno-pie -c addr.c -o addr.o
$ llvm-objdump-21 -r addr.o | sed -n '/.text/,/^$/p'
0000000000000001 R_X86_64_32  huge
$ ld -o addr addr.o
ld.bfd: addr.o:(.text+0x1): relocation truncated to fit:
  R_X86_64_32 against symbol `huge'

Two different outcomes for the same mistake.32 was chosen because the value did fit at compile time, so the link succeeded; at a different layout it would not have, and the 32 form would have truncated instead of reporting. 32S reports it as a hard error. The two types compute the same arithmetic and differ only in what they do when it does not fit — which is a format encoding a diagnostic policy in a numeric suffix.

A Real Example

Limit 3 is where a linker stops patching and starts writing code. x86-64 handles an out-of-range call by inventing a stub — a five-byte jmp instruction placed near the caller, with the far address written into it:

   the call site wanted:          e8 <32-bit displacement>
   the displacement does not fit.

   the linker writes instead:      the CALL stays, but now
                                   points at a nearby stub:

     ffffffff <displacement to stub>   call  stub
     e9 78 56 34 12                  jmp   0x12345678   <-- the real target

The relocation did not fail. It caused the linker to allocate space, emit an instruction, and patch the original field to point at the new instruction. That is qualitatively different from every other relocation in the table, and it is why a linker is not a patcher.

This is not a corner case you will meet only on absurd programs. The x86-64 code models exist precisely to control when it happens:

  -mcmodel=small     everything within 2GB, one instruction
  -mcmodel=medium    data via a register, calls still ±2GB
  -mcmodel=large     everything through registers, no
                     range assumption at all

And it is a per-target affair with no common mechanism. On AArch64 the ±4GB reach of ADRP pushes the problem out to 16GB away, and the fix there is a range-extension thunk. On PowerPC the linker can rewrite the instruction to use a 16-bit displacement field when the target is near. Every architecture invented its own answer to “the displacement did not fit”, and none of them resemble each other, because each is a consequence of that architecture’s available instruction space.

This is also the place where the previous concept’s AArch64 double-relocation stops looking like a cost and starts looking like a solution. AArch64 pays two relocations on every reference to get a wider range; x86-64 pays one relocation to get a narrower range, and then pays for out-of-range references with extra code. The trade is the same trade, taken at different points on the curve.

Try It Yourself

$ cd courses/reloc/assets/samples
$ ./build_samples.sh 2>&1 | sed -n '/F8/,$p'
$ xxd -l 8 far.o

Then test the limits against reality, which is where the numbers stop being theoretical:

  1. What is the actual size of the text segment in
     /bin/ls? How much of the ±2GB budget does a
     PIE binary on this machine actually use?

  2. A PIE is loaded at a random base. What is the
     LARGEST address the relocation engine could ever
     have to write on a 64-bit machine? If that is
     under 2GB, when does the stub case ever fire?

  3. Write a C file whose .text is deliberately enormous
     (>2GB is impractical, so reason instead). What
     breaks first: the PC32 displacement, or the
     32-bit absolute form? They are different limits.

  4. Compile with -mcmodel=large and diff the
     relocations. Which group from the previous concept
     grows, and by how much per reference?

Question 2 has the answer that makes the whole limit look different. A user-space process on Linux has an address space of 47 bits at most, and a PIE’s text is mapped near the top of that. So the practical worst case for a displacement is not 2GB of text — it is the distance from a high mapped address down to a low one, which is bounded by the user-space limit and comfortably under 247. The ±2GB limit is real and enforced, and on a modern 64-bit system with a normal-sized program it essentially never fires. The stub machinery exists for enormous binaries, 32-bit address spaces, and unusual layouts — not because everyday programs need it.

Connect

This closes the vocabulary module, and the three concepts make one descending argument. One Relocation, or Two said the vocabulary follows from reach. Why There Are So Many Kinds organised the set and named the historical accident inside REX_GOTPCRELX. This one supplies the arithmetic behind both — 32 bits signed, 231 bytes of reach, and the three ways a tool responds when that is not enough.

Into the PIE module the connection is about which limit you are actually near. The Flags, and Which One Is Which Kind and Does the Executable Actually Move? both concern a limit that is not the ±2GB one: what matters for a PIE is whether the base moves at all, and the answer is measured there. Range is the linker’s problem; relocation is the loader’s problem, and a PIE exercises the second while barely touching the first.

Back to the object files course, the connection is the loop closing on its central image. The Hole established that an object file’s relocations are a to-do list with zeros in it, and The Addend Lives in the Bytes established that the addend is separate. This concept is the arithmetic that makes both necessary: S - P cannot be known at compile time because P is the address of the field, which does not exist until layout is done. The zero is not a placeholder for laziness; it is the only representable value for a quantity that is undefined.

And the place where the three-way classification earns its keep is Applying Them Yourself, which closes the course. The exercise there is to write a relocation applier, and these three limits are exactly the specification it has to satisfy: add the symbol value and the addend, subtract the field’s own address for a PC-relative form, and refuse rather than truncate when a value does not fit. A patcher that gets those three rules right handles every relocation in the ELF x86-64 table except the ones where the linker has to create something — and those are the ones the concept above showed you are stubs, not patches.