Back to course

A Hole and a Record

20 minfoundationsModule 1: Why Object Files ExistContinues from obj-intro
20 min · Module 1: Why Object Files Exist · Foundations

Why This Matters

Last concept: an object file knows where its holes are. This one is about what a hole physically is, and the answer is stranger than "a zero".

Consider a call on x86-64. The instruction that calls a function directly is five bytes: a one-byte opcode followed by a four-byte relative displacement. The CPU computes the target as address of the next instruction + displacement. That is not a location, it is an offset from somewhere that is only meaningful once the code is placed — and at this point nobody knows where the code goes.

So the compiler writes zero into those four bytes and emits a relocation saying these four bytes are not a displacement yet. Not "are probably zero". Not "are a placeholder you may ignore". They are wrong, deliberately, and the file says so in a table the linker reads before it does anything else.

Get this one concept right and the rest of the format is bookkeeping. Get it wrong — treat a relocation as an optimisation hint, or as a patch to apply at compile time — and you cannot write a linker at all, which is where this collection is heading.

A Simple Model

A relocation record, at its most abstract. Every format in this course carries these same four ingredients, whatever it calls them:

a relocation answers four questions: 1. WHERE which bytes are wrong 2. HOW MUCH how many bytes are wrong 3. HOW what arithmetic turns them right 4. WHOM relative to which symbol and one ELF record answers them in exactly that order: u64 r_offset WHERE offset into the section's contents u64 r_info HOW+WHOM type in the high 32 bits, symbol index in the low 32 i64 r_addend an extra constant, usually 0 in an object 24 bytes. That is the whole record.

Question 3 is the one people skip, and it is the reason there are forty relocation types per architecture rather than one. The arithmetic is not always addition. For the call above it is S + A - P, where S is the symbol's final address, A the addend, and P the place being patched — because the field holds a relative offset and must have the instruction's own address subtracted out. For a 64-bit pointer in a data section it is just S + A, absolute, with no P anywhere.

So "how" is a small enum, and the enum is per-architecture — R_X86_64_PC32 is not the same enum as R_AARCH64_JUMP26, and neither resembles IMAGE_REL_AMD64_REL32 or X86_64_RELOC_BRANCH. Four names for the same idea, three incompatible numbering schemes. This is the subject of the relocation tables concept; the point here is that the concept of "how" is why a linker needs an architecture plugin.

Why the record lives outside the bytes it describes. A relocation could be stored in a header at the front of the section, as a table of offsets. All three formats instead store it in a separate section — .rela.text in ELF, a list inside the section header in COFF, a relocation entry in Mach-O. The reason is practical: a tool streaming through .text wants the code bytes contiguous and cache-friendly, and interleaving variable-length records would make that impossible. The cost is that you cannot find a hole by scanning the bytes — you have to join two tables by offset.

The Actual Detail

One relocation, from this course's 1952-byte demo_elf.o, out of the four against .text:

  OFFSET           TYPE                     VALUE
  0000000000000004 R_X86_64_PC32            global_counter-0x4
  000000000000000a R_X86_64_PC32            uninitialised-0x4
  0000000000000021 R_X86_64_PLT32           external_fn-0x4
  0000000000000033 R_X86_64_PC32            message-0x4

Take the first and answer all four questions from that one line.

And now the part that surprises everyone, visible in that one line: the -0x4 at the end is not part of the symbol name.

  global_counter-0x4     is really:  global_counter, with the
                            contents at .text+4 currently holding -4

  the disassembler prints it that way so you can see the value
  that will be ADDED to the contents. The real field is -4.

  why -4?  because the instruction is 4 bytes long, and the
            displacement is measured from the END of the
            instruction. So a correct displacement already has
            the instruction's own length subtracted out.

So the four bytes at .text+4 are minus four, not zero, and the linker adds S and gets the right answer. The addend lives in the bytes, and it is a real number that the compiler computed — this is the addends concept, and it is where most first attempts at an object-file writer go wrong.

What the call looks like, and why PLT32 is different

Offset 0x21 is a call to external_fn, a symbol defined in neither this file nor any file it knows about. Its type is R_X86_64_PLT32, not R_X86_64_PC32 — and the difference is a promise about the future, not a different calculation.

  R_X86_64_PC32     patch 4 bytes, S + A - P
  R_X86_64_PLT32   patch 4 bytes, S + A - P   AND tell the linker
                                             this might need a
                                             PLT entry

  the arithmetic is IDENTICAL. The extra bit is a hint: "if I
  end up calling into a shared library, build me a PLT stub and
  point at that instead."

  and COFF has no such distinction in an object file at all --
  demo_coff.o's call is a plain IMAGE_REL_AMD64_REL32, the same
  record as its data reference.

Same arithmetic, different bookkeeping, and one format has no concept of it. That is worth sitting with, because it is the first evidence of something that runs through the whole course: a relocation type is not only an instruction to patch bytes, it is also a piece of information for the linker's later decisions. A linker that implements only the arithmetic and ignores the hint will produce a binary that crashes on the first call into a shared library.

The same four bytes, three formats

And the same reference, as each format records it — which is the comparison that makes the concept stick:

  ELF     r_offset=0x04  r_info=(4<<32)|sym  r_addend=0
          a 24-byte record in its own section, .rela.text

  COFF    10 bytes, stored in the section header:
            VirtualAddress  0x000000000004
            SymbolTableIndex  0x0000000C
            Type              IMAGE_REL_AMD64_REL32
          no addend field AT ALL -- implicit, always

  Mach-O  8 bytes in a scattered list:
            r_address  0x04          an OFFSET, not a section address
            r_symbolnum  0x0C
            r_pcrel=1  r_length=2  r_extern=1  r_type=X86_64_RELOC_SIGNED
          width and arithmetic in FLAG BITS, not a type number

Three records, 42 bytes total, describing one four-byte hole. And Mach-O's is the most compact because it packs the arithmetic and the width into single bits — r_pcrel and r_length — rather than enumerating every combination as a type number. That is a genuine design trade, not an oversight: an enum costs nothing to read but grows without bound, while bit fields stay fixed and push the work onto the reader. A linker author needs both approaches, and needs to know which one each format chose before writing the arithmetic.

A Real Example

What this costs in practice, measured. Take the one line of C that produces a hole, and follow it through to the instruction bytes:

demo.c, one line: return helper(a) + private_counter + global_counter + uninitialised; becomes, in demo_elf.o, inside .text: becomes this, in demo_elf.o, inside .text -- the real disassembly, byte for byte: 0000 <compute>: 0: 01 ff addl %edi, %edi 2 bytes 2: 03 3d 00 00 00 00 addl (%rip), %edi 6 bytes ^^^^^^^^ 8: 8b 05 00 00 00 00 movl (%rip), %eax 6 bytes ^^^^^^^^ e: 01 f8 addl %edi, %eax 2 bytes 10: 83 c0 03 addl $0x3, %eax 3 bytes 13: c3 retq 1 byte 0020 <call_out>: 20: e9 00 00 00 00 jmp 0x25 5 bytes ^^^^^^^^ 0030 <use_data>: 30: 48 8b 05 00 00 00 00 movq (%rip), %rax 7 bytes ^^^^^^^^ the four ^^^^^^^^ markers sit at .text offsets 0x04, 0x0a, 0x21 and 0x33 -- which are EXACTLY the four offsets the relocation records named. Each one is the trailing 4-byte disp32 of a RIP-relative instruction: 6 bytes: 03 3d | 00 00 00 00 opcode 2, disp 4 5 bytes: e9 | 00 00 00 00 opcode 1, disp 4 7 bytes: 48 8b 05 | 00 00 00 00 REX, opcode, disp 4 and notice what is NOT there: no relocation for private_counter. It is a file-local int initialised to 3, so the compiler folded it into the immediate at 0x10 -- addl $0x3. Only a static's value can be folded this way; a global's cannot, because another translation unit may change it.

The hole is in the middle of a seven-byte instruction, not after it. That single fact is the reason this concept is harder than it looks and the reason instruction encodings are a prerequisite for writing a linker. A relocation is not "a pointer in the code" — it is a field at a specific bit offset inside an instruction encoding, and the linker must know the encoding to know it is allowed to write there.

Now the AArch64 version of the same source, which is the comparison that makes the cost concrete:

  x86-64 ELF, 4 relocations in .text:

    0004  R_X86_64_PC32             global_counter
    000a  R_X86_64_PC32             uninitialised
    0021  R_X86_64_PLT32            external_fn
    0033  R_X86_64_PC32             message

  AArch64 ELF, 8 relocations in .text -- SAME SOURCE:

    0000  R_AARCH64_ADR_PREL_PG_HI21   global_counter
    0004  R_AARCH64_LDST32_ABS_LO12_NC global_counter
    0008  R_AARCH64_ADR_PREL_PG_HI21   uninitialised
    000c  R_AARCH64_LDST32_ABS_LO12_NC uninitialised
    0020  R_AARCH64_JUMP26             external_fn
    0024  R_AARCH64_ADR_PREL_PG_HI21   message
    0028  R_AARCH64_LDST64_ABS_LO12_NC message
    (+ one more, for .rodata)

Eight records where x86-64 needs four, for identical C. Nothing about the language, the source, or the linker changed. AArch64's adrp instruction carries a 21-bit page delta and its paired load carries a 12-bit offset within the page, in two different encodings — so one reference needs two fixups, one per field.

Why this is the most important finding in the module. A learner who assumes "a relocation is a reference" will build a data structure with one entry per symbol reference, and it will be wrong on every AArch64 object — producing a linker that silently mis-resolves half its references on the most common server architecture on Earth. The correction is one sentence long and changes the shape of the code: a relocation is a reference to a field, not to a symbol. Two fields of one reference are two relocations, and nothing in the file tells you they are related. The pairing is a convention the instruction encoder knows and the linker must honour, which is why this course spends a module on encodings before it spends one on writing a linker.

And a practical detail that bites anyone writing a tool: relocation offsets are not always sorted. In demo_elf.o and demo_coff.o they ascend. In demo_macho.o the four .text relocations come out as 0x37, 0x26, 0x0e, 0x08 — descending, which is a real Mach-O invariant rather than an artifact of this compiler. A tool that binary-searches a relocation list, or streams it once assuming monotonicity, produces correct-looking output on two formats and quietly wrong output on the third.

Try It Yourself

$ llvm-objdump-21 -r -d courses/obj/assets/samples/demo_elf.o
$ llvm-objdump-21 -r -d courses/obj/assets/samples/demo_arm64elf.o

Check Your Understanding

Without looking back: an ELF object has one R_X86_64_PC32 record at offset 4 of .text, against symbol global_counter, with r_addend = 0. The four bytes at .text+4 currently read 00 00 00 fc. What do those four bytes mean, what will the linker compute, and what is the relationship between that computed value and the symbol's final address?

Apply It

You are writing the relocation application pass of a linker. It reads the ELF records for demo_arm64elf.o and, for each one, computes S + A - P and writes the low 32 bits. On x86-64 objects it is correct. On AArch64 objects it links without error and every call goes to a wrong address, while data references are sometimes right. What is the model of "a relocation" that your implementation assumed, and what is the smallest change to the data structure that fixes it without making the x86-64 path slower?

The general habit: a data structure shaped around the simplest case will be silently wrong on the general one, and the tell is a bug that links cleanly and fails at run time. Key on the thing the format actually varies — here, the field — not on the thing that happens to be simple.

Connect

A relocation is a work order deferred to link time, and that is the same shape as an ELF relocation in a linked binary — same four questions, different answers. The ELF course teaches the executable side, where r_addend is meaningful and the symbol's address is already known. Here the addresses are unknown and the arithmetic cannot even be set up. Learning the record once and meeting it in both places is the efficient order, and the object version is the harder one because it is the one that has to be designed rather than consumed.

The COFF and Mach-O courses teach the same record for their own formats, and the comparison here is what makes them stick. COFF's 10-byte relocation lives inside the section header rather than a section of its own, and has no addend field at all — the addend is always implicit and always in the bytes. Mach-O's 8-byte relocation packs the arithmetic and width into flag bits. Three placements, three ways of saying "how", and one conclusion worth carrying: a format's relocation design is a statement about what its linker needs to be fast, because all three of these choices exist to let the linker walk a section's relocations without branching on a type number.

That connects straight to the architecture section. AArch64's doubling is not a format quirk; it follows from an instruction encoding that splits an address into a 21-bit page delta and a 12-bit offset. And that is the same fact the architecture course teaches as the x86-64/AArch64/RISC-V encoding spectrum — variable-length versus fixed 32-bit versus fixed-with-compressed. The object file's relocation vocabulary is downstream of the instruction set's encoding choices, and once you see that, the forty relocation types stop looking like arbitrary enumeration and start looking like a direct transcription of which fields each instruction leaves for the linker to fill.

The "hole and a record" idea is also the collection's recurring shape, and it is worth naming as a pattern rather than a coincidence. A relocation is a hole plus a record. A PE base relocation is a hole plus a record. A PLT entry is a hole filled lazily plus a record. A GOT slot is a hole filled by the loader plus a record. In every case the record says what would make the hole right, and in every case the arithmetic is deferred to whoever can finally know the answer. A learner who has internalised this pattern will find dynamic linking largely self-explanatory, because dynamic linking is the same trick applied one step later, to a file that is already finished.

One practical note, because it is the kind of thing that costs an afternoon. The four bytes you inspected are little-endian on x86-64, so 00 00 00 fc is -4 and not 0x000000fc. Every one of these formats is little-endian on this target, which is not a property of the formats — it is a property of the targets, and it is why the JVM class file course opens with "everything here is big-endian, including the magic number" as a contrast. The one field that tells you the byte order is the magic number at offset 0, and you read it before you read anything else. The ELF, COFF and Mach-O courses each cover this for their own format.

Next: one source, three formats — the whole specimen set compared, to separate what an object file must have from what each format merely chose.