Back to course

Weak and Undefined-Weak

18 minintermediateModule 4: Merging and SelectionContinues from obj-archives
18 min · Module 4: Merging and Selection · Intermediate

Why This Matters

Here is C code that is the single most useful idiom in systems programming, and it looks like it should not compile:

  extern int optional_hook(int) __attribute__((weak));

  ... the whole of probe() is these three lines ...

      if (optional_hook)                  /* may be NULL -- this is the idiom */
          return optional_hook(5);
      return -1;

Testing a function pointer for null is legal C and it is the standard way to write an optional feature. The declaration says the symbol may be absent, and the if is the code saying “I know it might be.” Now look at what the object file has to record for that sentence to be true. The symbol is not defined here, and the linker must not treat its absence as an error. That is a third kind of symbol, distinct from both “defined” and “wanted”, and the three formats record it in three unrelated ways.

Measured on the same machine, same source, three targets. The weak definition and the undefined-weak reference side by side:

                    ELF                  COFF                        Mach-O
  weak definition    Bind = WEAK         the NAME is rewritten:       n_desc = 0x0080
                     (st_info nibble)    .weak.maybe.default.uses_weak   N_WEAK_DEF
                                         NO weak bit exists

  undefined weak     Bind = WEAK         NAME rewritten:              n_desc = 0x0040
                     + Ndx = UND        .weak.optional_hook.default.   N_WEAK_REF
                                         probe, Section = ABSOLUTE      + n_type = 0x01
                                         (-1), Value = 0                (N_EXT | N_UNDF)

Read the COFF column twice, because it is the surprise. There is no weak flag anywhere in a COFF object. No bit, no storage class, no aux field. The entire mechanism is that the symbol's name is rewritten to something that cannot possibly collide with anything another object would define — and the rewrite even embeds the name of the function that referenced it, which is the detail that will make you stop and reread the specification.

Three teams, one contract, and the answers are placed in three different tables: ELF in a binding nibble inside st_info, Mach-O in the n_desc field of a symbol, and COFF nowhere at all. That last one is the finding this concept is built around, and it is the clearest case in the course of a format solving a problem by using a field that was already there.

A Simple Model

Two questions, and it is worth keeping them apart because the formats answer them with different mechanisms and a reader who merges them will get both wrong.

Q1 MAY THIS DEFINITION BE OVERRIDDEN? A weak DEFINITION says: if anyone else defines this name too, use theirs and throw mine away. It is a statement about conflict resolution. Q2 IS IT AN ERROR IF NOBODY DEFINES IT? An undefined WEAK says: no. Resolve me to zero and carry on. It is a statement about the link SUCCEEDING. They are independent. A weak definition whose name is never defined elsewhere is just a definition. An undefined-weak reference is only ever a reference.

And Q2 has a consequence that is the whole reason the feature exists, and it is worth being concrete about the C semantics, because it is what makes the resulting binary safe rather than merely linkable:

the linker resolves the undefined-weak symbol to the address 0. so if (optional_hook) is if (0), which is FALSE &optional_hook is (void*)0 the code that dereferences it is on a path the author proved is unreachable, because the test is right there. The guarantee is only as good as the test. Nothing stops a program calling it unconditionally, and then the CPU jumps to address 0 and the process dies.

So the format's contribution is precisely one bit of information — “absence is not an error, absence is zero” — and everything else about safety is the programmer's. That is worth stating because it is the difference between a feature that is a language guarantee and one that is a linker behaviour. It is a linker behaviour. A linker that ignored the bit would report an error, and a program written against the feature would stop building — loudly, which is the good failure mode. A linker that honoured the bit but resolved the symbol to something non-zero would be far worse, and nothing in the object file would let a tool detect it.

Mark the model as a model. “Resolve to zero” is the behaviour on the targets measured here. It is not a universal law of the formats: a linker that must produce a position-independent binary cannot simply write the absolute address 0 into a field meant to hold an address, and the correct answer there is a relocation against the load base with an addend of zero. The contract is “absence is a defined value, not an error”; what that value is depends on whether the output is position-independent, and Position Independence is where that becomes visible. This is the third time in the course that a simple-looking constant turns out to be a position-dependent value, and the pattern is worth noticing as a pattern.

The Actual Detail

ELF: one nibble, used twice

Both cases from the staged specimens. w_elf.o defines maybe weakly; uw_elf.o references optional_hook without defining it:

  $ readelf -sW w_elf.o
     3: 0000000000000000  4 FUNC   WEAK  DEFAULT   2  maybe

  $ readelf -sW uw_elf.o
     4: 0000000000000000  0 NOTYPE WEAK  DEFAULT  UND  optional_hook

So ELF uses the high nibble of st_info — the binding — and that is the whole mechanism, for both questions. The undefined-weak case is then just the same nibble plus st_shndx == SHN_UNDEF, and the two are genuinely independent: a weak definition has a real st_shndx, an undefined-weak reference has SHN_UNDEF and a weak binding. Four combinations, and all four are meaningful.

The cost of this design is a three-level dependency, and it is the same one the COMMON concept found. Because SHN_UNDEF is what makes a weak reference may-be-zero, a reader that checks the binding without checking the section index will treat a strong undefined symbol as a weak one and silently resolve it to zero. Getting “absence is not an error” right requires reading st_shndx and st_info together, and there is no single field that answers it.

Mach-O: two bits, and one of them is a different question

  decoded from the nlist_64 entries by hand

  w_macho.o
    [0] _maybe           n_type=0x0f  sect=1  n_desc=0x0080  N_EXT N_SECT N_WEAK_DEF
    [1] _uses_weak       n_type=0x0f  sect=1  n_desc=0x0000  N_EXT N_SECT

  uw_macho.o
    [0] _probe           n_type=0x0f  sect=1  n_desc=0x0000  N_EXT N_SECT
    [1] _optional_hook   n_type=0x01  sect=0  n_desc=0x0040  N_EXT N_UNDF N_WEAK_REF

Mach-O gives the two questions different bits: 0x80 for “my definition may be discarded” and 0x40 for “do not error if I am unresolved”. That is cleaner than ELF's scheme in one respect — each bit answers exactly one question, so there is no inference from a combination — and it is what the COMDAT concept found Mach-O using for its entire group mechanism, with no section involved at all.

Note also n_type = 0x01 on the undefined-weak entry: bit 0 is N_EXT and bits 1–3 are N_UNDF. So “undefined” here is a value of the type field, not a sentinel in a separate field — the same trick the relocation record's r_extern bit plays, and a reminder that a format with spare bits in an existing field rarely adds a new field.

COFF: no mechanism, only a name

Now the interesting one. w_coff.o defines maybe weakly. Here is every symbol in the file:

  Name: .weak.maybe.default.uses_weak
  Value: 0
  Section: .text (1)
  BaseType: Null (0x0)
  ComplexType: Function (0x2)
  StorageClass: External (0x2)      the SAME as any normal function
  AuxSymbolCount: 0

  Name: uses_weak
  Value: 16
  Section: .text (1)
  ...StorageClass: External (0x2)

There is no weak flag. The storage class is IMAGE_SYM_EXTERNAL, exactly as for a strong function, and the section is the ordinary .text with no IMAGE_SCN_LNK_COMDAT bit. The only difference in the entire file is that the name is .weak.maybe.default.uses_weak instead of maybe. And the undefined-weak case is the same trick:

  Name: .weak.optional_hook.default.probe
  Value: 0
  Section: IMAGE_SYM_ABSOLUTE (-1)    not UNDEF(0): an absolute 0
  StorageClass: External (0x2)

Three parts, and each one is doing work:

And the undefined-weak case is the most elegant accident in the course. Section = IMAGE_SYM_ABSOLUTE (-1) with Value = 0 is a structural encoding of “the address zero”.Symbol Tables, Compared found that ABS is a genuinely third thing: the value is a constant, not an address and not an offset, and adding a section base to it produces a pointer that was not intended. Here that semantic turns out to be exactly right — the may-be-zero contract is an absolute zero, expressed by choosing the one section index that means “absolute” and setting the value to zero. No new field, no new flag: the format says “this is the constant 0” by pointing at the category that already means constants.

And the cost of the COFF design is severe enough to name. Because the rewrite is a name rewrite, a tool cannot ask “is this symbol weak?” — it can only notice that the name begins with .weak. Any tool that reads COFF symbol names for a different purpose is broken by this: a symbol browser shows .weak.maybe.default.uses_weak instead of maybe; a rewriter that tries to rename maybe finds no such symbol; and “which function is weak?” becomes a string operation on a name that was never meant to be parsed. Compare ELF, where the answer is one nibble, and Mach-O, where it is one bit — and understand that this is why the other two are better, and that COFF got there by having a symbol table with no spare room.

What actually happens: strong beats weak, silently, and the bytes stay

The behaviour matters more than the encoding, so here it is end to end. w_elf.o has a weak maybe; w2_elf.o has a strong one. Merge them:

  $ ld -r w_elf.o w2_elf.o -o rw.o
  (silent -- no warning, no diagnostic)

  $ readelf -sW rw.o | grep maybe
       8: 0000000000000020  4 FUNC  GLOBAL  DEFAULT  1  maybe

  The symbol is now GLOBAL -- promoted from WEAK -- and
  points at offset 0x20.  There is no second `maybe`.

  $ llvm-objdump-21 -d rw.o
    0000000000000000 <.text>:
         0: 8d 47 ff       leal  -0x1(%rdi), %eax      the WEAK body
         3: c3             retq
        10: e9 00 00 00 00  jmp   uses_weak+0x5        the CALLER
        20: 8d 47 64       leal  0x64(%rdi), %eax      the STRONG body
        23: c3             retq
        30: b8 65 00 00 00  movl $0x65, %eax           `other`

Look at offset 0. The weak definition's bytes — 8d 47 ff c3, the x - 1 version — are still in the merged section, at the start, with no symbol pointing at them and nothing that will ever execute them. The symbol table no longer contains the weak maybe at all. Weak defeat removes the symbol; it does not remove the code.

That is the finding, and it is not a curiosity. The only way to reclaim those five bytes is for the weak definition to have been in a section of its own — which is exactly what a COMDAT group provides, and exactly what plain __attribute__((weak)) does not. The COMDAT concept's group section and this concept's weak symbol are the same mechanism with and without the section boundary, and this is the measurement of what the boundary is worth.

And the numbers behind that: w_elf.o's .text is 21 bytes and w2_elf.o's is 22, but the merged .text is 54. 21 + 22 = 43, so 11 bytes were added — and 5 of them are the dead weak body, with the rest alignment padding. Both inputs' code is present in the output.

Finally the undefined-weak case, verified by running it:

  $ ld -r uw_elf.o -o ruw.o      # nothing defines optional_hook
  (silent)

  $ readelf -sW ruw.o | grep optional
       8: 0000000000000000  0 NOTYPE  WEAK  DEFAULT  UND  optional_hook
  ^^^^^^ still WEAK, still UND, and that is not an error

  and the C idiom, linked and run both ways:

  $ gcc t.o -o t        &&  ./t
  hook ABSENT -> the weak reference resolved to 0, and the test was false

  $ gcc t.o h.o -o t2   &&  ./t2
  hook present, returned 55

Same source, same binary layout, two different answers depending on whether another object supplied the symbol. And note what the first link did not say: no warning, no note, no marker in the output. An undefined-weak that stays unresolved is completely invisible in the linked product. Which is a real operational hazard, because “the feature silently did not get compiled in” and “the feature is correctly disabled because nothing provided it” are the same binary. A tool that wants to tell them apart has to look at the input objects, not the output.

A Real Example

The three mechanisms, side by side, scored on the things an implementer has to get right. Every row of it is measured on the specimens in this course's assets/samples/:

                        ELF         COFF              Mach-O
  ----------------------   ----------   ---------------   -----------
  where the answer is     st_info      the symbol's      n_desc
                          high nibble   NAME

  bits needed               1 nibble      0              1 (def)
                                                     1 (ref)

  can a tool ask          yes           no -- only by    yes
  "is this weak?"                      parsing a name

  two independent
  questions                one nibble    one name        two bits
                          + st_shndx    prefix          N_WEAK_DEF
                                                     N_WEAK_REF

  undef-weak encoded as    st_shndx =    Section =       n_type =
                          SHN_UNDEF      ABSOLUTE(-1)    N_UNDF(0)
                          + WEAK         Value = 0
                                         structural

  what happens to a        the symbol   both copies      both copies
  defeated definition     is dropped;   coexist under    coexist
                          the BYTES     rewritten       unowned
                          stay

Three rows deserve a second look, because each is a design lesson rather than a fact.

“Can a tool ask is this weak?” — ELF yes, Mach-O yes, COFF no. That is the strongest argument against the COFF design, and it is not about the linker at all. A linker only needs to resolve names, and COFF's renaming achieves that. Every other tool — a symbol browser, a coverage tool, a rewriter, a debugger, a linker that wants to report which definitions were discarded — is worse off. The general principle: encoding a property in a name makes the property unavailable to anyone who is not willing to parse the name, and the number of tools willing to parse a name is small and falling.

“What happens to a defeated definition” — ELF keeps the bytes, COFF keeps both copies, Mach-O keeps the bytes. All three leak, and only the COMDAT-group variant does not. So a toolchain that cares about this has to arrange for weak definitions to be group members, which is a policy layered on top of the format's mechanism. Which is the honest summary of weak symbols in every format: they make a name optional, they do not make its storage free.

And “undef-weak encoded as” — three different structural answers to the same question. ELF says “the value is not a value; it is a constraint on placement.” Mach-O says “the symbol is a promise, and a promise has its own type bit.” COFF says “this is a constant, and the constant is zero, and I already have a category for constants.” Given a choice between inventing a sentinel and finding a category that already means the right thing, COFF found the category — and that is the most elegant thing in this module, and also the reason its design has the sharpest edges.

Try It Yourself

$ cd courses/obj/assets/samples
$ readelf -sW w_elf.o uw_elf.o | grep -E 'WEAK|maybe|optional'
$ llvm-readobj-21 --symbols w_coff.o uw_coff.o | grep -A3 'Name: .weak'
$ llvm-objdump-21 -d --syms uw_macho.o

Check Your Understanding

Without looking back: a COFF object contains a weak definition of maybe, and you are writing a symbol browser. Its symbol table shows .weak.maybe.default.uses_weak with StorageClass = IMAGE_SYM_EXTERNAL and the section being the ordinary .text. There is no weak flag, no storage class for weak, and no COMDAT bit. Meanwhile the equivalent ELF object has st_info >> 4 == STB_WEAK and the Mach-O object has n_desc == 0x80. Why can your browser answer the question for two of the three formats and not the third, and what is the cost of the third format's choice?

Apply It

You are writing a size-optimising linker for ELF. It drops unreferenced sections before resolving symbols, which is a large win on C++ binaries with thousands of .text._Z...comdat sections. It works. Then a program that uses __attribute__((weak)) for an optional platform feature silently stops providing that feature, with no diagnostic, and only on one machine. Given what the previous concepts measured, what is happening, and what is the smallest change that fixes it without giving up the size win?

The general habit: weak, COMMON and COMDAT all break a naive reachability sweep, because each is a reference whose satisfier is not obvious from the relocation graph. Fix the phase ordering once rather than adding a special case per mechanism.

Connect

This concept closes the selection module, and the three concepts in it are worth reading together as one argument. COMDAT answered “which of two definitions survives”. The archive answered “which of three hundred objects is loaded at all”. This one answers “what if nothing supplies it at all” — and the three together are the complete set of answers a linker has to a symbol that is not simply there. The shape of all three is the same: the format gives the linker permission to make a decision the language otherwise forbids, and records enough for the decision to be made the same way twice. That is the design pattern, and it recurs in every format in the collection.

The connection to Symbol Tables, Compared is where the COFF ABS encoding connects up. An undefined-weak reference in COFF is a symbol with SectionNumber = IMAGE_SYM_ABSOLUTE and Value = 0 — and that concept found ABS to be a genuinely third category, distinct from both “undefined” and “defined in a section”, with the warning that adding a section base to it produces a pointer that was not intended. Here is a case where the format's own semantics are exactly right and the warning's consequence is the feature. The may-be-zero contract is an absolute zero, so the format says it by pointing at the category that already means constants. Compare ELF, which had to invent SHN_COMMON for its version of the same idea, and Mach-O, which spent a whole section on it. Three answers again, and the one that reused an existing category is the shortest.

There is a direct link to COMMON and the ABI Break that is more than thematic. Both mechanisms exist because C has a symbol whose definition is not in the file you are looking at — COMMON because a tentative definition may be satisfied by another object, undefined-weak because the definition may never exist. And both had the same fate in the same decade: the format recorded a decision each producer made locally, and nothing in the format said the two decisions were incompatible, so GCC 9 and GCC 10 broke every prebuilt library on every Linux distribution. The lesson generalises past COMMON: a format that lets producers record an optionality decision needs a way to make two different optionality decisions interoperate, or the decision has to be made by the language instead. C++ did that for inline functions; C never did for tentative definitions, and the bill came due.

On the operational side the invisible-unresolved case is worth connecting to something outside the course entirely, because it is a general engineering shape. A feature that is silently disabled when its provider is missing looks exactly like a feature that is correctly disabled when nothing provides it, and no amount of inspecting the output binary will tell you which happened. Every build system that supports optional components has this problem, and the general solutions are the same three: make the absence loud (a warning that survives), record the decision in the output so it can be inspected later, or make the provider's presence a compile-time fact rather than a link-time one. Weak symbols give you none of the three, which is why they are used for exactly the cases where a silent default is the desired behaviour — and why a platform feature built on them should have a way to report “this was not compiled in” that does not go through the weak symbol.

Finally, the dead-bytes finding connects straight to the module that follows. Writing One From Scratch is about the smallest complete object file, and it turns out that the hand-built one needs a concept this module has just finished supplying: a weak symbol is how you make a definition optional, and an undefined symbol is how you make a reference deferred — and the object file being built there has one of each, an undefined helper and a PLT32 against it. Module 3 gave the arithmetic and Module 4 gave the selection rules; the next module asks you to write both down.