Back to course

Symbol Tables, Compared

22 minintermediateModule 2: Anatomy, ComparedContinues from obj-sections
22 min · Module 2: Anatomy, Compared · Intermediate

Why This Matters

The previous concept ended with a lesson about dictionary keys. This one is the other half of it, because the symbol table is the other place a format hands you something that looks like a record and is not one.

A symbol table looks like the simplest structure in any of these formats: a flat array of named things. In ELF that is very nearly true. In COFF it is not true at all, and a reader that believes it is will read fourteen garbage entries and believe them. That is not a hypothetical: this course's own 1098-byte specimen has 26 records of which only 14 are symbols.

And underneath that, all three formats answer the same four questions in genuinely different ways. What is my name? Where am I? Am I defined here, or do I want somebody else to define me? How visible am I? COFF answers the third question with two fields that must agree, and ELF answers it with one field that means something different from what it looks like.

A Simple Model

The four questions and the three record layouts. The sizes are from this course's specimens, all 64-bit targets:

question ELF COFF Mach-O ------------------ --------------- ------------------ --------------- what is my name u32 -> .strtab 8 bytes, inline u32 -> __stringtab OR strtab offset where am I u64 value u32 value u64 value which section u16, 0=UNDEF i16, 0=UNDEF u8, 0=UNDEF -1=ABS -2=DEBUG am I defined implied by StorageClass u8 nlist flags section != 0 MUST ALSO be read type bits how visible u8 st_info StorageClass u8 n_desc binding + type (same field!) (n_type) record size 24 bytes 18 bytes 16 bytes how many .symtab PointerToSymbol- n_syms sh_size Table + count in the header

Two structural observations before the details.

COFF uses one field for two questions.StorageClass answers both am I defined and how visible am I, which is why those two questions are entangled in every COFF reader ever written. ELF splits them cleanly into st_info's high and low nibbles. That is not a simplification, it is a genuine improvement — and it is also the reason a COFF reader's bug reports are so hard to read, because the field that is wrong is doing two jobs.

And Mach-O's record is the smallest at 16 bytes — it packs binding, type and visibility into two bit-fields — and it is the format where a name is an offset into a per-segment __stringtab, so the reader must first find the right segment's string table. Smallest record, most work per record. That trade recurs throughout these formats and is worth watching.

The Actual Detail

All 26 records of demo_coff.o's symbol table, decoded. The table starts at 0x211, the records are 18 bytes each, and the string table follows at 0x3e5:

  IDX  NAME                          HOW         VALUE  SECT        CLASS
  [ 0] .text                         inline      0x0     .text(1)     STATIC
  [ 1] >                            --          0x-36fd4e6c  .text(1)  ENDOF_FUNC   <- NOT A SYMBOL
  [ 2] .data                         inline      0x0     .data(2)     STATIC
  [ 3]                              --          0x-206078b  .data(2)  ENDOF_FUNC   <- NOT A SYMBOL
  [ 4] .bss                          inline      0x0     .bss(3)      STATIC
  [ 5]                              --          0x0     .bss(3)      ENDOF_FUNC   <- NOT A SYMBOL
  [ 6] .rdata                        inline      0x0     .rdata(4)    STATIC
  [ 7]                              --          0x672d789d  .rdata(4)  ENDOF_FUNC   <- NOT A SYMBOL
  [ 8] ??_C@_05CJBACGMB@hello?$      strtab+73   0x0     .rdata(4)    EXTERNAL
  [ 9] .rdata                        inline      0x0     .rdata(5)    STATIC
  [10]                              --          0x-d04b58a  .rdata(5)  ENDOF_FUNC   <- NOT A SYMBOL
  [11] .debug$S                      inline      0x0     .debug$S(6)  STATIC
  [12]                              --          0x7831d036  .debug$S(6) ENDOF_FUNC  <- NOT A SYMBOL
  [13] .llvm_addrsig                 strtab+45   0x0     /45(7)       STATIC
  [14] e                             strtab+0    0x0     /45(7)       ENDOF_FUNC  <- NOT A SYMBOL
  [15] @feat.00                      inline      0x0     ABS(-1)      STATIC
  [16] compute                       inline      0x0     .text(1)     EXTERNAL   type=0x20
  [17] global_counter                strtab+18   0x0     .data(2)     EXTERNAL
  [18] uninitialised                 strtab+59   0x0     .bss(3)      EXTERNAL
  [19] call_out                      inline      0x20    .text(1)     EXTERNAL   type=0x20
  [20] external_fn                   strtab+33   0x0     UNDEF(0)     EXTERNAL
  [21] use_data                      inline      0x30    .text(1)     EXTERNAL   type=0x20
  [22] message                       inline      0x8     .data(2)     EXTERNAL
  [23] message_bytes                 strtab+4    0x0     .rdata(4)    EXTERNAL
  [24] .file                         inline      0x0     DEBUG(-2)    103      naux=1
  [25] demo.c                        inline      0x0     UNDEF(0)     0          naux=0

Fourteen of those twenty-six records are not symbols. Records [1], [3], [5], [7], [10], [12] and [14] are auxiliary data belonging to the section symbol above them, and you can see the tell: every one has a garbage Value like 0x672d789d and a StorageClass of 0. A reader that does not honour NumberOfAuxSymbols does not fail loudly. It prints twelve nonsense symbols with nonsense addresses, and the nonsense looks exactly like real data.

This is the concept's central trap, and it is nastier than the section-name collision because there is no duplicate key to collide with. The mechanism is one byte: each section symbol carries naux=1, and the following record is payload. Section symbols need an auxiliary record because a section's size and relocation count are not in the symbol, and COFF's designers put them here rather than in the section header — where, as the previous concept showed, they also are. So the same facts are in the file twice in COFF, once in the section header and once in a symbol-table auxiliary record, and a reader that trusts one has no reason to check the other. Cross-checking them is the cheapest available validation of a COFF reader, and this course's specimen makes it a one-line test: the Value in each aux record should equal the section's rawsize.

The name field, and both mechanisms in one record

The 8-byte name field is a union, and which half is live depends on whether the first four bytes are zero. Both cases appear above:

  inline:   the 8 bytes ARE the name, NUL-padded

      b'.text\x00\x00\x00'              -> ".text"
      b'.debug$S'                      -> ".debug$S"    exactly 8, no NUL
      b'@feat.00'                      -> "@feat.00"
      b'compute'                       -> "compute"

  strtab:  the first 4 bytes are ZERO, the last 4 are a
           DECIMAL offset into the string table at 0x3e5

      strtab+4   -> "message_bytes"
      strtab+18  -> "global_counter"
      strtab+33  -> "external_fn"
      strtab+45  -> ".llvm_addrsig"     (the /45 escape from
                                         the previous concept)
      strtab+59  -> "uninitialised"
      strtab+73  -> "??_C@_05CJBACGMB@hello?$AA@"

Eight bytes of inline name is generous compared to ELF's four-byte offset, and COFF still needs the escape. The reason is visible in record [8]: ??_C@_05CJBACGMB@hello?$AA@ is 27 characters. It would not fit, so it goes in the string table — and that is the same table, at the same offset scheme, that the previous concept found behind the /45 section name. One string table, two uses, and a reader that has already walked it for section names can walk it again for symbol names.

So COFF's name handling is: inline if it fits, string table if it does not, and no uniqueness requirement in either case. ELF is: always the string table, four bytes, and the name is never in the record at all. Mach-O is: always a string table, four bytes, but the table you need depends on which segment the symbol is in — so a Mach-O reader needs the section table before it can read a single name.

Defined, undefined, absolute, and debug

Now the third question, and COFF's answer is the interesting one. Look at the SECT column:

  section  0  = UNDEF   the symbol is wanted, not defined
  section -1  = ABS     the value is not an address at all
  section -2  = DEBUG   a debug-directory pseudo-section
  section >0  = a real section index

  and it is NOT sufficient on its own:

  [20] external_fn   sect=UNDEF(0)  class=EXTERNAL   <- a real want
  [25] demo.c        sect=UNDEF(0)  class=0          <- NOT a want
  [15] @feat.00      sect=ABS(-1)   class=STATIC     <- absolute
  [24] .file         sect=DEBUG(-2) class=103        <- the file name

Records [20] and [25] both have section 0 and both are "undefined" by that field alone, and only the StorageClass distinguishes an actual undefined reference from the file-name record. That is the coupling the model predicted, in a real file. A COFF reader that asks only "is the section number zero" will treat demo.c as an undefined symbol and try to resolve it.

And ABS is a genuinely different concept that the other two formats spell differently. An absolute symbol's Value is not an address and not an offset — it is a constant. @feat.00 at ABS with value 0 means "the constant 0". A linker that adds a section address to it produces a pointer, which is not what it meant.

ELF, and the same four questions

For contrast, ELF's table on the same source. readelf -s reports 12 entries against COFF's 14 real symbols, and the difference is instructive rather than a discrepancy:

  ELF has, that COFF does not:
    one STT_FILE entry naming demo.c        (COFF: .file + aux)
    STT_SECTION entries for .text and
      .rodata.str1.1 ONLY -- not for .data,
      .bss or .rodata, despite .data having
      a relocation against it

  and the reason is demand: a section symbol
  exists because something NAMED that section.
  The .data relocation targets the variable
  "message", so nothing needs a .data section
  symbol and none is emitted.

So COFF emitted seven section symbols and ELF emitted two, from the same input. Not a bug in either, and not a disagreement about the format — a difference in policy about when a convenience entry is worth 18 bytes. A tool that assumes "every section has a symbol" allocates seven where ELF needs two; a tool that assumes "symbols and sections correspond one to one" breaks on ELF. Neither assumption is safe, and both directions of mismatch occur in files produced by the two most common toolchains on Earth.

A Real Example

What a linker does with all this, and the three jobs that are harder than they look. A symbol table is read four times over the course of a link, and each pass wants something different:

pass 1 COLLECT which symbols does this object DEFINE? -- needs: section > 0 (ELF) or section > 0 AND a sane class (COFF) -- COFF: must skip aux records first pass 2 WANT which symbols does it WANT? -- ELF: section == 0, non-local -- COFF: section == 0 AND the class that means "external reference". Record [25] is section 0 and is NOT one. pass 3 RESOLVE match wants to definitions. -- the whole point. Needs both sides to agree on VISIBILITY, or you get a local that satisfies a global. pass 4 RELOCATE what address does this symbol have? -- ELF/COFF: section address + value -- Mach-O: value IS the address

Pass 3 is where COFF's one-field design bites hardest. A static symbol and an EXTERNAL symbol both have a section number, and only the storage class says which is visible outside the file. Get it wrong in the permissive direction and a local symbol satisfies a global reference, so two files' private helpers collide and one silently wins. Get it wrong in the strict direction and a legitimate global reference fails to resolve. Both are silent, and neither shows up until the program misbehaves at run time — which is the recurring theme of this entire course.

Pass 4 is the previous concept's finding restated at the symbol level, and it is where the Mach-O difference bites hardest because it is the one that looks like a unit error and is not. Mach-O's Value is already an address; ELF's and COFF's are offsets. Adding a section base to a Mach-O value double-counts. That is the same bug as global_counter reading 0x48 instead of 0, arriving through a different door, and it is why a reader should convert to a common internal shape immediately rather than carrying a format-native value around.

And the honest cost, which is a real design tension rather than a bug in either format. COFF's 8-byte inline name means a reader can get a name without touching the string table, which is a genuine speed win for a linker that resolves millions of symbols. ELF's always-offset means the section-name and symbol-name tables are independent, so you can walk either without the other, which is a genuine modularity win for a tool that only wants one of them. Both are optimising for different access patterns, and neither is wrong. The cost of COFF's choice is the aux-record interleaving, which is pure overhead paid by every reader including the ones that only want names. The cost of ELF's choice is a second string table in the file. A format is a set of bets about which access pattern is common, and reading one properly means understanding which bet its designers made.

One more thing worth knowing before you write a reader, because it is the kind of detail that costs an afternoon. COFF's Type field packs a base type in its high nibble and a derived type in its low one, and the three functions here all have Type = 0x20. That is 0x2 << 4 — base type 2, which is function. The data symbols have Type = 0. So the field distinguishes code from data, and it is the only place in a COFF object that does so directly — ELF uses STT_FUNC versus STT_OBJECT, and Mach-O uses an N_TYPE mask. Three vocabularies for "is this code or data", all answering a question a linker needs to answer when it decides whether a symbol is executable.

Try It Yourself

$ readelf -s demo_elf.o
$ llvm-readobj-21 --symbols demo_coff.o
$ llvm-nm-21 demo_macho.o

Check Your Understanding

Without looking back: a COFF reader sees record [20] with SectionNumber = 0 and StorageClass = 2, and record [25] with SectionNumber = 0 and StorageClass = 0. What is each one, what does the reader get wrong if it tests only the section number, and what is the equivalent single-field test in ELF that avoids the whole problem?

Apply It

You are writing the symbol-collection pass of a linker. It walks each object's symbol table, and for every record with a nonzero section index it publishes the name as a global definition. On ELF and Mach-O it is correct. On COFF it publishes seven section names as if they were global symbols, and two of those names collide with real symbols elsewhere in the link, so a reference to a genuine .data gets bound to a section. The link succeeds and the program reads the wrong address. What is the single defect, and what is the smallest change that fixes it without special-casing COFF by name?

The general habit: normalise at the reader boundary, then let everything downstream be format-blind. One field doing two jobs in the file becomes two clean values in your program, and the special case never reaches the code that matters.

Connect

The ELF course's symbol table and symbol binding concepts teach ELF's version of this, and the COFF course teaches COFF's. What neither can do is show you both at once, which is what makes the two-field-versus-one-field argument land: once you have seen a format that needs a pair of fields to answer a question another format answers with one, the ELF design stops looking like obvious simplicity and starts looking like a deliberate four-byte-per-symbol purchase. That is the recurring lesson of the triangulation method, and this concept is its cleanest example.

The aux-record trap has a close relative in a format you have already met, and it is worth naming because the shape is identical. A COFF string table and an ELF constant pool both use the same trick of a length field meaning "and here are that many more records" — and the JVM course's finding 16 is the same species: an annotation value that is a 2-byte index where you expected 4 bytes of inline data, caught only because the parse did not balance. In every case the failure mode is identical: an array whose length is not in its own shape, so a reader that trusts the shape desynchronises and produces confident garbage.

Mach-O's leading underscore and its single-letter type vocabulary connect to the Mach-O symbol table concept, and this concept supplies the why for both. The underscore is there because Mach-O's runtime ABI prefixes C symbols, and the type letter collapse is there because the section table already distinguishes read-only from zero-fill. Both are consequences of a Mach-O object carrying more structural information than ELF's does — which is also why its symbol records are the smallest of the three and its sections the largest. The trade is visible in one comparison and it is not a quality judgement.

And the pass-4 address bug is the third appearance of the Mach-O-provisional-address finding, which makes it a pattern rather than an incident. It appeared in the triangulation concept as a table of numbers, in the segments concept as the reason Mach-O objects carry addresses at all, and here it is as a concrete arithmetic error a linker will make. Three sightings of the same asymmetry, from three directions, and each one made it more concrete than the last. That is how a fact becomes something a reader gets right rather than something they have heard about.

One connection forward, because it is where this table is actually used. Everything in this concept serves symbol resolution — matching wants to definitions, and deciding which of several competing definitions wins. That is the whole of the second course in the collapsed linking roadmap, and the fields measured here are its inputs: the StorageClass that COFF entangles, the binding nibble ELF separates, the weak and common cases neither of these specimens contains. A reader that gets this concept right can already parse every input a resolver needs; it just cannot yet decide anything. That boundary is the honest place to stop, and the next module is where the fixups that reference these symbols get taken apart.