Back to course

One Instruction Called -Bsymbolic

21 minadvancedModule 2: Building a LibraryContinues from dyn-export
21 min · Module 2: Building a Library · Advanced

Why This Matters

A library that defines a function and calls it. The most ordinary code there is, and interposition means the call is not necessarily the library’s own.

/* self.c */
int helper(int x){ return x * 3; }        /* defined HERE */
int entry(int x){ return helper(x) + 1; } /* and called HERE */

/* selfmain.c */
extern int entry(int);
int helper(int x){ return x * 1000; }     /* the program tries to interpose */
int main(void){ printf("  entry(2) = %d\n", entry(2)); return 0; }

Build the library twice, once with the flag:

$ clang -fPIC -shared                  -o libself.so     self.c
$ clang -fPIC -shared -Wl,-Bsymbolic   -o libself_sym.so self.c
$ clang -o t_plain    selfmain.c ./libself.so     -Wl,-rpath,'$ORIGIN'
$ clang -o t_symbolic selfmain.c ./libself_sym.so -Wl,-rpath,'$ORIGIN'
$ ./t_plain; ./t_symbolic
  entry(2) = 2001
  entry(2) = 7

2001 against 7. The library’s own helper returns 2*3, plus one, so 7. The program’s returns 2*1000, so 2001. -Bsymbolic made the library win its own argument — and it is one instruction.

A Simple Model

Find the difference between the two libraries, because it is smaller than the flag name suggests.

$ for lib in libself.so libself_sym.so; do llvm-objdump-21 -d --no-show-raw-insn $lib | sed -n '/<entry>/,/^$/p' | grep call done libself.so callq 0x1030 <helper@plt> libself_sym.so callq 0x1100 <helper> ONE INSTRUCTION. callq <helper@plt> the call goes to the PLT, which jumps through a GOT slot the LOADER fills. indirect, and the loader is free to put anyone there. callq <helper> the call goes straight to the address the LINKER computed. no PLT, no GOT slot, no loader decision. There is nothing left to interpose.

That is the whole mechanism, and it is a relocation difference rather than a policy. The reference to helper normally gets R_X86_64_PLT32 — a PC-relative reference that the loader resolves through a slot, precisely so it can choose a different answer. -Bsymbolic makes the linker resolve it at link time instead, to a definition it can see in the file it is producing. The relocation type changes; the search never happens.

And the thing the name hides, which is the part worth the concept:

-Bsymbolic binds references to symbols THE LIBRARY ITSELF DEFINES. not: references to symbols in other libraries references to symbols it does not define anything about what it EXPORTS a library calling libc's memcpy: -Bsymbolic does nothing. there is nothing of the library's to bind; the symbol belongs to libc and the loader will decide where memcpy lives. a library calling its own internal helper: -Bsymbolic binds it, because the library can see the definition at link time. the flag is "bind MY symbols", read precisely. not "bind symbols".

The first attempt to measure this in this course got it wrong in exactly that way — it used a library calling a function from a different library, and the flag changed nothing at all: identical disassembly, identical behaviour, still 110. That was correct behaviour and it is the teaching point, so both experiments are below.

Does It Actually Work?

First, that .dynsym is untouched — because this is a binding flag, not an export flag:

$ for lib in libself.so libself_sym.so; do
    printf "  %-16s dynsym: " $lib
    llvm-nm-21 -D --defined-only $lib | awk '{printf "%s ", $3}'; echo
  done
  libself.so       dynsym: entry helper
  libself_sym.so   dynsym: entry helper

Identical. Both libraries still export helper, and both are still interposable by anything that wants them. -Bsymbolic does not hide the symbol; it changes what the library’s own call binds to. Those are two different questions and the previous concept was about the first one.

Now the negative case, which is the one that makes the rule precise. A library that calls another library:

$ clang -fPIC -shared -Wl,-Bsymbolic -o libb_sym.so mid.c -L. -la
$ llvm-objdump-21 -d --no-show-raw-insn libb_sym.so | sed -n '/<mid_value>/,/^$/p' | grep call
  1114:  callq  0x1030 <lib_value@plt>
                       ^^^^^^^^^^^^^^ STILL THROUGH THE PLT

$ ./prog3        # main2 + libb_sym.so, same interposition
  prog's lib_value()=100   libb's view of it=110

Identical to the non-symbolic build, and the interposition still happens.lib_value is defined in liba.so, not in libb, so -Bsymbolic had nothing to bind and correctly did nothing. If you reach for this flag to stop a library from being interposed on, and the interposition is coming from a symbol the library does not define, it will not work and nothing will tell you.

Which brings up the two flags you actually want, and the difference between them:

  -Bsymbolic            every reference the linker can
                       resolve WITHIN this library is bound
                       locally. one instruction per call site.

  -Bsymbolic-functions   the same, but only for FUNCTIONS.
                       references to DATA still go through
                       the GOT and remain interposable.

  -fno-semantic-interposition
                       a COMPILER flag, and different in
                       kind: it tells the compiler that a
                       call to a function in the same
                       translation unit may be turned into
                       a direct call, because no interposition
                       will be allowed. it is finer-grained
                       than -Bsymbolic and it changes the
                       code the COMPILER emits, before any
                       linker is involved.

  measure the difference on a self-referential function:
    -fno-semantic-interposition  produces the SAME single
    direct callq as -Bsymbolic, and does it for a case
    -Bsymbolic cannot reach: a call the linker never sees
    as a relocation at all, because it was inlined away.

Which of the three you want depends on a question only you can answer: do you care about being interposed on, or only about being inconsistent? A library that wants to be replaceable should use neither. A library whose internal helpers must be reliable wants both.

A Real Example

The case where a library is genuinely broken by interposition, which is rare, real, and worth recognising when you see it.

  the shape: a library with INTERNAL STATE that
  its exported functions both read and write.

    static int counter;              /* file-static */
    int  bump(void){ return ++counter; }
    int  read(void){ return counter; }

  both bump() and read() are in .dynsym.

  a program that defines its OWN bump() -- for any
  reason, including a completely innocent one -- now has
  read() return 0 forever, because read() calls the
  program's bump(), which touches a different counter.

  this is not a security bug. it is a program that is
  correct on the build machine and wrong everywhere
  else, and the symptom is "a library function always
  returns the initial value", which is a miserable thing
  to debug.

And note what does not fix it: static on counter already guarantees the library’s two functions share one variable. The problem is not the variable, it is the function call between them, which goes through the PLT and can be redirected. Shared state protected by a file-static variable is still reachable through a replaceable function. That is a genuinely counter-intuitive corner, and it is the clearest argument for -Bsymbolic being a correctness tool.

The performance side, which is real and modest and worth stating honestly rather than overselling:

  the direct call saves:
    - the PLT stub: one indirect jump
    - the GOT load: usually already in L1, so the
      dependent-load cost is small in practice
    - for a hot leaf function called in a loop, this is
      measurable; for a function called once, it is not.

  the honest summary: -Bsymbolic is a correctness and
  predictability flag that happens to be slightly faster.
  Build it for the first reason. If you are choosing
  between "measurable speedup" and "my library behaves
  like itself", take the second every time.

And the cost, which nobody mentions and which is the real reason the flag is not the default. It removes a feature. A library built with -Bsymbolic can no longer be interposed on itself, which means:

  you cannot use LD_PRELOAD to replace one of its
  internal functions in order to instrument it. profilers,
  tracers, allocators and sanitizers all work by
  interposing.

  you cannot patch a bug in an internal function
  without a source rebuild.

  the library becomes opaque to the ecosystem of tools
  that depends on being able to substitute a function.

  which is a legitimate choice for a leaf utility and a
  bad one for a widely-deployed library. there is no
  universally correct answer, and pretending otherwise
  is how people end up with libraries that cannot be
  profiled.

Try It Yourself

$ cd courses/dyn/assets/samples
$ ./build_samples.sh 2>&2 | sed -n '/D4/,/D6/p'
$ python3 crosscheck.py 2>&2 | sed -n '/\\[4\\]/,/\\[6\\]/p'

Then map the boundary of the flag, which is the useful skill:

  1. For each of these, does -Bsymbolic change the
     instruction, and why?

       (a) a library calls its own function
       (b) a library calls another library's function
       (c) a library calls a libc function
       (d) a library calls through a function POINTER it
           was handed
       (e) a library calls an IFUNC resolver

     (d) and (e) are the interesting ones: neither is a
     call the linker can resolve, so neither can be
     affected by a LINK-time flag.

  2. Add -Bsymbolic-functions to the -Bsymbolic build and
     diff. Which references changed and which did not? (A
     data reference cannot become a direct access, so
     only the function calls should differ.)

  3. Write the counter example from the previous section
     and run it with and without -Bsymbolic. Then WITHOUT
     any flag but with the program's bump() renamed. (All
     three behave differently and only one is right.)

  4. Take a real library with a known interposition
     problem and try to fix it with -Bsymbolic. Count the
     instrumentation tools that stop working. Is the
     trade worth it? (Answer for yourself; there is no
     general answer and anyone who claims otherwise has
     not tried to profile it.)

  5. The one to end on: build libself.so, then patch its
     PLT stub by hand with a debugger or objcopy so the
     call goes somewhere else. It is the same class of
     change -Bsymbolic makes, done after the fact. What
     does that tell you about how much of the loader's
     flexibility is a decision rather than a necessity?

Exercise 1 is the one that gives you the rule rather than the anecdote, and the answer is worth predicting before you run it. -Bsymbolic can only affect a call the linker resolves, which means a call that becomes a relocation in the object file. A call through a function pointer does not — the target is a runtime value — and an IFUNC resolver is chosen by the loader for its own reasons. So (a) changes, (b) and (c) do not, and (d) and (e) cannot be reached by a link-time flag at all.

Exercise 5 is the philosophical one, and it is the reason to understand this flag rather than just memorise it. The difference between callq <helper@plt> and callq <helper> is five bytes and a relocation type, and it is the difference between a system where tools can substitute functions and one where they cannot. Most of what looks like a technical necessity in a linker is a decision somebody made, and this is one of the clearest.

Connect

This closes the building module, and it is the exact complement of the previous concept. That one asked can anyone see this symbol and answered with .dynsym contents. This one asks does this call go through the search and answers with a relocation type. Hiding removes a name from the search space; this removes a call from it. A library that hides its internals still has exported functions calling each other through the PLT, and those are still candidates — which is why one concept is not enough.

The connection to the Relocations course’s vocabulary concept is the mechanism, and it is the clearest payoff in the course. That concept grouped the relocations into the linker’s arithmetic and the loader’s deferred work, and noted that R_X86_64_PLT32 exists so a call can be indirect. This concept is what that indirection buys and what it costs. A PLT32 reference is a request to the loader: you decide where this function lives. -Bsymbolic is a way of declining to make the request — and it is a relocation-type change, not a policy setting, which is exactly the lesson that concept taught about the PIC violation where a diagnostic could name the type.

And the connection to the symbol-resolution course’s PLT concept closes a loop. That course decoded the six-instruction PLT entry, the link_map push, the resolver, and the return address adjustment. Everything it decoded is the machinery that -Bsymbolic turns off for a given call site. The resolver still exists, still works, and is still used for every reference that did not decline — but this one call never reaches it, and the whole sequence collapses to a direct jump.

Forward into the run-time module, the subject shifts from which binding is chosen to when. Lazy, Eager, and What Actually Differs is about the timing of the same decision, and it contains a finding that refines what the sym course measured: -z now changes no code, exactly as that course said, but it makes .got.pltdisappear. Two flags, one subject, opposite directions: one removes a call from the search, the other removes the wait.

One connection outward, and it is the sharpest framing in the course. Interposition is a feature: it is how a program replaces a library function without cooperation, how LD_PRELOAD works, and how profilers have been built for thirty years. -Bsymbolic is how a library opts out of that feature. The whole course is a series of mechanisms for controlling one search, and this is the point at which the searcher gets a vote. A system where every library opts out would be faster and completely unprofilable, and a system where none does is flexible and occasionally baffling. Neither is the right answer, which is why the flag exists rather than a default.

render_lesson_js(&mut page)