Back to course

No Segments, and Why

16 minintermediateModule 1: Why Object Files ExistContinues from obj-triangulate
16 min · Module 1: Why Object Files Exist · Intermediate

Why This Matters

Every executable you have run has two tables describing its own contents, and they say different things. A linked ELF has a section table grouping bytes by purpose — .text, .data, .rodata — and a program header table grouping them again by how to map them, with permissions attached.

An object file has the first and not the second. readelf does not hedge about it:

  $ readelf -lW demo_elf.o

    There are no program headers in this file.

That is a strange thing to find, because the two tables are so often taught together that the absence reads like a bug. It is not. The second table answers a question that an object file cannot yet answer, and working out which question is worth a concept in its own right — because it is the cleanest available answer to the question the ELF course raises when it contrasts sections and segments, and it generalises to COFF and Mach-O without modification.

A Simple Model

Two questions, two tables, and an object file can only answer the first:

QUESTION 1: what bytes are in this file, and how do they group by purpose? answered by: the SECTION table .text .data .bss .rodata who asks: you, and a linker deciding what to keep in an object: YES QUESTION 2: how should a loader map those bytes, with what permissions, at what addresses? answered by: the SEGMENT table (ELF program headers) LOAD 0x000000 R LOAD 0x001000 R E LOAD 0x002000 R LOAD 0x002e00 RW who asks: the kernel, at exec time in an object: NO -- and it CANNOT be answered yet

Three things in that second table are worth pausing on, because together they are the whole reason it cannot exist in an object.

Permissions. A segment says R, R E or RW. That is a statement to the kernel's page-table code, and deciding it requires knowing which sections ended up adjacent after every object has contributed its own. A compiler compiling one translation unit cannot know whether its .rodata will end up next to another object's .rodata or next to that object's .data.

Addresses. A segment carries a virtual address. No address is knowable until layout is done, which is the first concept's point and the reason the object has none.

And the page granularity itself. Segments are page-aligned by construction, so the fourth LOAD below starts at file offset 0x2e00 but loads at memory address 0x3e00. Those differ by exactly one page of padding, inserted so the segment can be made writable without a page holding read-only data becoming writable too. You cannot compute that offset until you know every section's final size and every neighbour's alignment.

So the absence is not a simplification — it is an impossibility. The segment table is not "the same table, left out for size". It is a table whose every field depends on information that does not exist until link time. A format that shipped one anyway would be shipping a table full of zeroes that the linker has to rewrite completely, which is strictly more work than not writing it.

The Actual Detail

The same program, as an object and as a linked executable, side by side. Both produced on this machine:

  $ clang -target x86_64-pc-linux-gnu -O1 -c -o exe.o exe.c
  $ clang -target x86_64-pc-linux-gnu -O1 -c -o other.o other.c
  $ clang -target x86_64-pc-linux-gnu -O1 -o exe exe.o other.o

                     OBJECT              EXECUTABLE
  size               1952 bytes          16136 bytes
  e_type             REL                DYN  (PIE)
  entry point        0x0                0x1040
  sections           16                 31
  program headers    0                  12

Four differences, and only the first is the one this concept is about.

Zero program headers, then twelve. And the four that matter for mapping:

  $ readelf -lW exe | grep -A1 LOAD

    LOAD  offset 0x000000  vaddr 0x00000000  filesz 0x5e8   R      0x1000
    LOAD  offset 0x001000  vaddr 0x00001000  filesz 0x181   R E    0x1000
    LOAD  offset 0x002000  vaddr 0x00002000  filesz 0x160   R      0x1000
    LOAD  offset 0x002e00  vaddr 0x00003e00  filesz 0x220   RW     0x1000
                                          ^^^^^^^^
                     the RW segment's file offset and its load
                     address differ by 0x1000 -- one page of padding

The first three segments have vaddr == offset and the fourth does not. The reason is exactly the reason the segment table could not be written by a compiler: the writable data must start on a page boundary in memory, and the bytes before it in the file are not page-aligned, so a page of padding goes in. That padding is a decision about layout, made during layout, by something that had to see every input.

The section count went up, not down

Sixteen sections in, thirty-one out. That surprises people who expect a linker to be subtractive, and the additions are the interesting part:

  sections the LINKER created, none of which any
  compiler ever emitted:

    .interp        the path to the dynamic linker
    .dynamic       the table the dynamic linker reads
    .got .got.plt  slots for addresses not yet known
    .plt           stubs for functions not yet callable
    .rela.dyn      relocations the LINKER could not resolve
    .rela.plt      and the ones for the PLT
    .init_array    constructors, in the order to run them
    .fini_array    destructors
    .symtab .strtab .shstrtab   the output's own symbol table

The linker is not only filling holes; it is generating structure that did not exist. A .plt is a code sequence the linker invents. A .got is a table the linker allocates. Those are not sections from the object, moved — they are new artefacts of the linking process itself, and a static linker creates some of them so the dynamic linker will have them later.

Which is the honest way to state the division of labour, and it corrects a tidy story that is easy to believe:

the tidy story (wrong): compiler produces sections, linker produces segments, so segments are "the linker's table" what actually happens: the compiler produces sections and holes the linker produces addresses, segments, AND more sections -- including ones that exist only so the RUNTIME linker can do its job later

The relocation table changed shape completely

Last measurement, and the one that closes the loop. In the object, four relocations against .text, all against symbols this file does not define. In the executable, zero relocations against external_fn, because other.o supplied it and the linker resolved them. What survives is a different species entirely:

  $ readelf -rW exe | head -6

    0000000000003e00  R_X86_64_RELATIVE      1120
    0000000000003e08  R_X86_64_RELATIVE      10e0
    0000000000004008  R_X86_64_RELATIVE      4008
    0000000000004018  R_X86_64_RELATIVE      2004
    0000000000003fc0  R_X86_64_GLOB_DAT   __libc_start_main@GLIBC_2.34

  none of these name a symbol defined in this file. RELATIVE
  just says "add the load base to the value already here".
  GLOB_DAT says "ask the dynamic linker for this one".

Symbol-relative relocations became base-relative ones. The linker resolved everything it could against the objects it was given, and what survived is only what genuinely cannot be known until run time — where a shared library will be mapped, and where libc is. That transition, from "ask the linker" to "add a base", is the entire difference between a relocatable file and a dynamically-linked one, and it is why the dynamic relocations are a separate subject from the object ones in this course.

And COFF and Mach-O agree completely on the absence, which is what makes this a fact about object files rather than about ELF. All three specimens, checked:

  demo_elf.o    ELF program headers:  0
  demo_coff.o   COFF: no base relocations, no directories
  demo_macho.o   Mach-O: ONE LC_SEGMENT_64 -- and its
                 segname field is 16 zero bytes. Unnamed.
                 vmaddr 0x0, vmsize 252, holding all 6
                 sections flat.

COFF's spelling is different — a PE has no segment table at all, because a PE is always a mapped image and its sections carry the virtual addresses directly — and Mach-O's is the interesting one, because Mach-O objects do carry an LC_SEGMENT_64 command and it is still not a mapping plan. That command exists, its size is 72 bytes, and its segname field is sixteen zero bytes: one unnamed segment holding all six sections flat, with vmaddr 0x0 and vmsize 252. The per-section segname fields still say __TEXT and __DATA, so the section table retains the grouping while the segment command declines to express it. It is a placeholder with the right shape, and a reader must know not to treat it as one.

A Real Example

Why this matters to someone building a compiler, which is where this course is going. Consider what a code generator must decide at emit time, and which of those decisions are available:

a code generator emitting an object file must decide: section assignment which section does this go in? ANSWERABLE. .text for code, .rodata for literals, and so on. section ordering where should .rodata go relative to .data in the output? NOT ANSWERABLE. The linker sorts to satisfy alignment and locality, and may merge them. section flags read-only? writable? executable? PARTIALLY. You can say "this is .rodata" and the linker works out the permissions -- but only because whole sections share a fate. addresses where will it be? NOT ANSWERABLE. Emit relocations. permissions per can this function have a page that page another function can also write? NOT ANSWERABLE, and you must not try.

The last row is the one that produces real bugs. It is tempting for a code generator to emit a segment table anyway, filled in with a guess, because then the file looks like a proper image and tools display it nicely. The result is a file that readelf renders with a plausible segment table and ld then either rejects or silently discards it, depending on the linker.

The trap, stated as a rule.Do not emit a table you cannot fill in correctly. A zero-filled segment table is worse than an absent one, because its absence is unambiguous to every tool and its presence is a claim. This is the same principle as the JVM course's finding that a Record attribute's presence is the declaration while its content is secondary, and as the ELF course's warning about e_shnum == 0 being a sentinel rather than a count. In every one of these formats, the difference between "absent" and "present but meaningless" is a distinction every reader has to implement, and it is always better to be absent.

Now the part that surprises people, and it is the practical reason the segment table belongs to the linker. Permissions are granted per segment, which in practice means per section-group, and that only works because the linker sorts sections into permission groups. Every .rodata from every object ends up on the same page with the same permissions, because they are all read-only. If a compiler emitted a function needing its own writable page, there would be nowhere to record that, and the segment table would need a per-section concept it does not have.

That is a real architectural decision with a real cost, and the PE format makes the same trade in a different place: a PE is always a mapped image, so it has no segment table and inherits the same constraint, with sections carrying IMAGE_SCN_MEM_READ and friends instead. One format defers the decision to link time and one makes it at compile time; both end up with page-granularity permissions, because that is what the hardware does.

And why a dynamic executable has segments at all, while a static one could in principle be laid out by a simpler scheme, is the hinge to the loading courses. A PIE's segments carry addresses that are not the final addresses — the loader maps them somewhere else and adds a base, which is exactly what the surviving R_X86_64_RELATIVE relocations are for. The segment table in the object has no base because the object is not mapped; the segment table in the executable has provisional addresses because the executable is mapped somewhere, just not somewhere final. One table, two reasons for its addresses to be wrong, and a different mechanism for each.

Try It Yourself

$ readelf -lW demo_elf.o
$ readelf -lW exe | grep -A1 LOAD

Check Your Understanding

Without looking back: the RWLOAD segment of a linked executable has file offset 0x2e00 and load address 0x3e00, while the three segments before it have vaddr == offset. Why does only the last one differ, and why could a compiler not have written that value even if it had wanted to?

Apply It

You are writing a static linker and you decide, for convenience, to emit a program header table into the executable as soon as you have laid out the first few sections — updating it as you go. It works. On inputs with an unusual section ordering it produces an executable that ld accepts, readelf renders with correct-looking segments, and the kernel refuses to run with permission denied. What is the invariant your incremental approach broke, and what is the general rule for a table whose fields depend on each other?

The general habit: a table whose entries are functions of a shared, still-changing quantity must be emitted after that quantity is final, never maintained alongside it. Incremental metadata that depends on a whole goes stale silently.

Connect

This is the ELF course's sections-versus-segments concept seen from the file that has only one of them, and that is the better order — because once you have seen a file with no segment table, the two-table model stops looking like a design choice and starts looking like two different questions. The ELF course can then answer "what does a segment do" against a file that demonstrates why it cannot exist yet, which is a much stronger footing than a definition.

Compare the PE, which made the opposite decision and has a genuinely instructive reason for it. A PE is always a mapped image — there is no such thing as a relocatable PE in the ordinary sense, because a PE's sections carry VirtualAddress and the loader honours them directly. So a PE has no program header table at all, and its section table does the segment table's job. ELF supports relocatable files and therefore needs two tables; PE does not, and therefore needs one. That is the cleanest available demonstration that a format's table count follows from its requirements rather than from taste.

The page-alignment padding connects to something with teeth in the security courses. A segment's permissions are a security boundary precisely because the hardware enforces them per page — and the padding is what makes that boundary land correctly. An R E segment means "this page is executable and not writable", and W^X is only meaningful because a page is the unit. Move to a byte-granular architecture and the entire model changes, which is a fact The Memory Hierarchy course makes concrete when it reaches page tables and TLBs.

The linker-generated sections are the bridge to the rest of the linking half of the chain, and it is worth naming the dependency explicitly. A .plt and a .got exist because a call to a function in a shared library cannot be resolved at link time — so the linker emits a stub and a table, and the dynamic linker fills them in later. That is dynamic linking, and it is the reason the linker is a program rather than a file filter. An object file is where the dynamic linker begins, and this concept is the last of the four that explains why the object file is shaped the way it is.

Two smaller connections worth keeping. The surviving R_X86_64_RELATIVE relocations in the executable are the same records with a different question — the dynamic relocation set is not a new format but a new set of answers to the same four questions from the second concept, with "whom" replaced by "the load base". And the e_type field, which reads REL here and DYN in the executable, is a one-value enum whose whole job is to tell a reader which of the two models applies — which is the same trick as the JVM's ACC_ENUM and the same reminder that a format needs one bit to say which world it is in.

That closes Module 1. The object file is now a known thing: what it is for, what a hole is, that three formats agree on the mechanism and differ on everything else, and why one of its two tables is necessarily absent. Module 2 opens the file up — the section and symbol tables compared field by field, where the three formats disagree most and where a tool author has the most to get wrong.