Weak Externals

19 minadvancedModule 5: Other Targets and Other SectionsContinues from coff-tls

19 min · Module 5: Other Targets and Other Sections · Algorithms

Why This Matters

Every symbol in Modules 1 and 2 had one of two states: defined here, or defined somewhere else and required to be. A reference to an undefined symbol is an error. That is the rule, and it is what makes a library's API meaningful — you can only call what somebody promised to provide.

But a great deal of useful code depends on a symbol that might be there. A program that uses a system's high-resolution timer will use one if the platform has it and fall back if not. A library that accelerates a hash function will use the platform's if it exists. A plugin host that supports an optional extension must be able to ask whether it is present, which is a question the linker has to be able to answer without failing the build.

That is what a weak external is: a reference that the linker is permitted to leave unresolved. It is a third state, and it is not a flag on the reference — it is a storage class on the symbol, which means it lives in the symbol table and needs an auxiliary record to say what "unresolved" should resolve to instead. The symbol table concept decoded the 18-byte record and its auxiliary chains; this concept is the case where the auxiliary record carries information nothing else in the file can supply.

A Simple Model

Produce one:

One function declared weak, and one caller that tests it before using it:

Line	Content
1	`extern int other_fn(void) __attribute__((weak))`
2	`int call(void)` whose single-expression body is `other_fn ? other_fn() : 0`

$ clang --target=i686-pc-windows-msvc -c weak.c -o weak.obj

And read the symbol:

$ llvm-readobj --symbols weak.obj
  Name: _other_fn
  Value: 0
  Section: IMAGE_SYM_UNDEFINED (0)
  BaseType: Null (0x0)
  ComplexType: Null (0x0)
  StorageClass: WeakExternal (0x69)
  AuxSymbolCount: 1
  AuxWeakExternal
    Linked: .weak._other_fn.default._call (17)
    Search: Alias (0x3)

Four things in that, and each one is a decision:

Field	Value	Means
`StorageClass`	`0x69` = 105	Not "external". A class of its own: this reference may go unresolved
`Section`	0 = undefined	As for any external. The weakness is additional to being undefined, not a replacement
`AuxSymbolCount`	1	An auxiliary record follows, and it is not optional
`AuxWeakExternal`	The answer to "unresolved, so what instead?"

The auxiliary record is the whole design. A weak reference with no fallback rule is not a decision, it is a wish, and a linker cannot act on a wish. So the record supplies one of three answers:

`Characteristics`	Name	If the symbol is missing
1	`Library`	Look for it in a library, as an ordinary external would be
2	`Alias`	Bind to the fallback symbol named in the same record
3	`AntiDependency` — reported by the reader as `Alias` here	Bind to the fallback, and treat a real definition as a conflict

All three name a fallback symbol in the second field, so the record is two symbol-table indices: a TagIndex naming the fallback, and a Characteristics saying how to treat it. The reader this course has available prints the characteristics value 0x3 with the label Alias, which is worth noting rather than glossing over — see the callout below.

A tool disagrees with itself, or with the specification. The value in the file is 0x3. The specification's three values are 1, 2 and 3, and 3 is IMAGE_WEAK_EXTERN_SEARCH_ANTI_DEPENDENCY. The reader prints it as Alias, which is the label for 0x2. So either the reader's table is off by one, or the producer emitted 2 and the reader is displaying 3, or the producer emitted 3 and the reader's label is wrong. The bytes are the authority and they say 03. What is not established here is which of the three labels is correct, because there is no second independent reader of this field on this machine to ask. So the concept states the value, states that the specification's third value is the anti-dependency case, and does not claim the reader's label is right. This is a smaller version of the readelf-versus-llvm adjudication in the loclists concept, and the discipline is the same: report the disagreement rather than picking a side.

The Actual Detail

The fallback symbol, and what it is not

The aux record names symbol 17. Look at what that is:

$ llvm-nm weak.obj
00000000 A .weak._other_fn.default._call

An absolute symbol — section 0, value 0 — whose name encodes the whole mechanism in one string:

.weak . _other_fn . default . _call
  |       |          |        |
  |       |          |        +-- a per-declaration suffix, so two weak
  |       |          |           references to the same function get
  |       |          |           different alias symbols
  |       |          +----------- "no library search" / default handling
  |       +---------------------- the symbol being made optional
  +------------------------------ "this is a synthetic alias, not code"

So the fallback is not a function. It is a synthetic symbol with a descriptive name and no storage, whose only job is to be the address the linker reports when the real symbol is absent. The address of a missing function is the address of a name. And because it has value 0 and no section, reading it as a location gives 0 — which is also why the calling code tests the pointer before calling it.

The three states, and what the caller does with each

Now the machine code, which is the part that makes the design make sense:

$ llvm-objdump -d --section=.text weak.obj
  call:
    0:  84 c0           test  %eax,%eax
    2:  74 03           je    7
    4:  ff d0           call  *%eax
    6:  c3              ret
    7:  31 c0           xor   %eax,%eax
    9:  c3              ret

Two instructions of test, then a call through a register, then a zero return on the other path. The source said other_fn ? other_fn() : 0, and the compiler emitted exactly that: the weak reference compiles to a load, a test, and a branch, because the symbol's value is not known until link time and might be zero.

Compare that with an ordinary external call, which compiles to a direct relative call to a known offset. The difference in the generated code is the entire observable effect of weakness: weakness costs a branch. That is a real cost on a hot path, which is why a library will mark a symbol weak only when the optional path is genuinely cold, and it is the reason the storage class is a deliberate choice rather than a free annotation.

What the linker does, and where it stops being a linker job

The resolution is a short algorithm, and one of its steps is not in the linker at all:

for each undefined symbol with StorageClass WeakExternal: if a definition exists anywhere in the link: bind to it (the normal case) else: bind to AuxWeakExternal.TagIndex apply AuxWeakExternal.Characteristics: Library -> also search the library archives Alias -> done; the fallback is the answer AntiDependency -> done, and flag a real definition as a conflict rather than a win

That is the whole mechanism, and it is entirely a link-time decision. So a weak external is a promise the linker keeps, not the loader and not the program — which is the useful thing to know when you are looking for the answer to "was this symbol actually present?".

It does not survive into the image as a weak reference. By the time a binary exists, the symbol is either a normal defined symbol or the fallback's address, and nothing in the image records that it was once weak. That has a direct consequence for reverse engineering: you cannot tell from a stripped binary whether a function pointer was optional or mandatory. A null test in the disassembly is evidence of a weak reference, because an ordinary call would not compile to one — but it is evidence, not proof, since a defensive null check on an ordinary pointer looks identical.

One more thing the record enables that is easy to miss, because it is the anti-dependency case. Marking a symbol weak does not only permit it to be missing; it can also mean "if this gets defined, something has gone wrong". That is the mechanism a platform uses to detect an application accidentally defining a symbol the platform was going to supply — the definition does not silently win, it is reported. It is a debugging aid expressed in the symbol table, and it is the reason the Characteristics field has three values rather than a boolean.

A Real Example

The three real uses, and what each looks like at the symbol-table level. They differ in which field carries the meaning, which is the part worth remembering.

Intent	What the caller must do	Symbol-table mechanism
Optional acceleration	Test the pointer, call if present	Weak, fallback = an absolute symbol; a branch in the code
Optional platform feature	Test the pointer, fall back if absent	Weak, `Characteristics = Library` so a library may still provide it
Detect an accidental definition	Nothing — it is a link-time diagnostic	Weak, `Characteristics = AntiDependency`

The middle row is the one with a subtlety worth spelling out, because it is the difference between "optional" and "optional but encouraged". Characteristics = 1 tells the linker to do an ordinary library search for the symbol. If no archive provides it, the fallback applies. So the symbol is still resolved to a real definition when one is available, and the weak class only comes into play at the point where the search has failed. The three values are not three strengths of the same thing; they are three different questions asked at three different moments — during library search, after it, and as a conflict check.

Now the tooling question, which is where a format like this usually gets misused. A tool that wants to know "is this symbol guaranteed present?" has three possible answers and must not conflate them:

"Is it DEFINED in this image?" -> read the symbol table of the image -> a weak reference resolved to a real definition is indistinguishable from an ordinary one. Answer: yes. "Was it DECLARED weak in the source?" -> not knowable from the image -> only from the objects, before linking "Does the program CHECK it before use?" -> not knowable from the symbol table at all -> only by reading the code

The middle and bottom blocks are the ones people get wrong, and they are wrong in the same direction: assuming the image remembers a property of the source that the linker resolved away. A weak reference is a link-time construct, and like every other link-time construct it leaves no trace in the output. A static analyser that wants to report "this call is to an optional function" has to read the code, find the null test, and reason about it — and even then it is inferring, not reading a fact.

That is the same conclusion the COMDAT concept reached from the other side. There, a checksum in the file was a promise the linker spent and a consumer could skip; here, a storage class in the file is a decision the linker makes and then forgets. Both are cases where the interesting information exists only before the link, and a tool that runs after it is working from a file that has already thrown the answer away. The link is a boundary, and anything you need to know about the program has to be recovered on the near side of it.

Try It Yourself

$ clang --target=i686-pc-windows-msvc -c weak.c -o weak.obj
$ llvm-readobj --symbols weak.obj | grep -A5 WeakExternal
$ llvm-nm weak.obj | grep weak

Build the three cases. Mark one symbol with __attribute__((weak)), one as a plain weak definition rather than an extern, and one with an explicit fallback. Compare the storage class, the AuxSymbolCount, and the Characteristics in all three. The definition case is the interesting one, because a weak definition is a promise the linker keeps against a duplicate rather than one it makes about a missing symbol.
Read the raw 10 bytes. The AuxWeakExternal record is two fields: a 4-byte symbol index and a 4-byte characteristics value. Find them at the right offset after the 18-byte symbol and confirm they are 11 00 00 00 and 03 00 00 00. Then you can adjudicate the reader's label dispute yourself, because you will be reading the specification value and the file's bytes side by side.
Link it both ways. Link weak.obj alone, and then with a second object that defines _other_fn. Compare the two images: the disassembly of call should be identical, and only the value the fallback resolves to should differ. That is the cleanest demonstration that weakness is a link-time property with no run-time cost beyond the branch.
Prove the branch is the cost. Compile a function calling a weak symbol without the null test — just call it — and compare the generated code. You will get a direct call and a link error or a crash, which is the point: the test is not defensive coding the programmer added, it is the mechanism, and removing it removes the mechanism.
Look for weak externals in a real binary. Take any Windows PE you have and look for .weak. symbols or StorageClass 105 entries. Most will have been resolved away, which is the point — but an unresolved one is a fascinating find, because it means a weak reference survived into the image.
Check the third value carefully. The specification's value 3 is the anti-dependency case and the reader calls it Alias. Work out which is right from the file's bytes and the specification, and if you can find a second tool that reads the field, compare all three. A one-byte disagreement about a three-valued enum is a good exercise in how much you should trust a single reader.

Check Your Understanding

Without looking back: an object has a symbol with StorageClass: WeakExternal (0x69), Section: 0, AuxSymbolCount: 1, and an aux record naming symbol 17 as the fallback with Characteristics = 3. Symbol 17 is an absolute symbol named .weak._other_fn.default._call at value 0. No object in the link defines _other_fn. What address does the linker give _other_fn, and what will a program that calls it do?

Apply It

You are auditing a shipped binary for a security review. Someone asks whether the application's use of a platform crypto function is safe, and you need to answer: is that function guaranteed present, or does the program handle its absence? You have the binary, the debug info, and the original object files. What can you actually determine, from each, and what can you not determine at all?

The general habit: when auditing a shipped artefact, ask which build stage consumed the information you need, and go get it from before that stage. A question the linker answered cannot be asked of the linker's output.

Connect

This is the payoff for a detail in the symbol table concept that could easily have been left as trivia: the 18-byte record has a one-byte NumberOfAuxSymbols field, and a value of 1 means "an auxiliary record follows, and it is not a COMDAT block". The symbol table concept decoded three auxiliary shapes; this is the fourth, and it is the one that is required rather than optional — a weak external with no aux record does not mean "no fallback", it means malformed.

That is the same relationship the COMDAT concept had, one level up. There, the aux record carried a checksum so a linker could merge two identical sections. Here it carries a fallback so a linker can survive a missing one. Both are the format answering the question "what should a consumer do when the expected thing is not there?" One answer is a comparison to perform, the other is a substitution to make, and neither can be defaulted because both are decisions with consequences.

The generated code closes the loop with relocations. An ordinary external call is a REL32 fixup against a symbol the linker guarantees exists, so it compiles to a direct call. A weak one is the same fixup against a symbol the linker may leave at zero, so it compiles to a load, a test and an indirect call. The branch in the instruction stream is the storage class, made visible. That is a satisfying thing to be able to see, and it is also the honest answer to how much weakness costs.

One concept left, and it is the mirror image of the TLS one. TLS carried information the loader needs, at run time, per thread. This next section carries information the linker needs, at link time, and which the linker throws away the instant it has read it — so the section is not in the output at all, and a reader has to know it existed to understand why something else is missing.

Next: Linker Directives in a Section.