Back to course

Lazy, Eager, and What Actually Differs

20 minadvancedModule 3: Loading at Run TimeContinues from dyn-dlopen
20 min · Module 3: Loading at Run Time · Advanced

Why This Matters

Build the same program twice and diff it. Most people expect a difference in the code, because “resolve everything at startup” sounds like it should produce different instructions.

$ clang -O2                        -o lazy     lazy.c -L. -llazy -Wl,-rpath,'$ORIGIN'
$ clang -O2 -Wl,-z,now         -o lazy_now lazy.c -L. -llazy -Wl,-rpath,'$ORIGIN' -Wl,-z,now

$ for f in lazy lazy_now; do
    printf "  %-9s .plt=%s  .got.plt=%-6s .got=%s  dyn entries=%s\n" $f \
      "$(readelf -SW $f | awk '/ \.plt /{print $6}')" \
      "$(readelf -SW $f | awk '/\.got\.plt/{print $6}')" \
      "$(readelf -SW $f | awk '/ \.got /{print $6}')" \
      "$(readelf -dW $f | sed -n 's/.*contains \([0-9]*\) entries.*/\1/p')"
  done
  lazy      .plt=000020  .got.plt=000020 .got=000028  dyn entries=28
  lazy_now  .plt=000020  .got.plt=       .got=000048  dyn entries=29

The .plt is byte-identical, and a whole section disappeared. The symbol-resolution course established that -z now changes no code, and that is exactly right. What that course could not show is the second half: .got.plt is gone, and .got grew from 0x28 to 0x48 to absorb it.

A Simple Model

What the two PLT slots do, because the difference is entirely about what a slot is for.

a PLT GOT slot holds one of TWO things, and the difference is one bit: the linker put a RESOLVER ADDRESS there the loader has not run yet. first time this call is executed, the slot is the "go to the resolver" address, and calling through it runs dl_runtime_resolve, which finds the real function, writes it into the slot, and jumps there. every SUBSEQUENT call goes straight through. the loader put the REAL ADDRESS there because -z now made it resolve everything before the program started. the slot has no other job, so it does not need its own section. which is why .got.plt vanishes under -z now: it exists to hold slots that have TWO states. a slot with one state is just .got.

So .got.plt is not “the GOT for PLT entries”; it is the part of the GOT that is not yet resolved. That is a much better way to hold it, and it explains the section disappearing exactly rather than approximately. The reloc course measured the default script’s rule for it — .got.plt : ... *(.got.plt) *(.igot.plt) — without knowing that on this toolchain the section can be empty.

And the decision itself, in the form the loader reports it:

DT_FLAGS BIND_NOW added under -z now DT_FLAGS_1 NOW the NOW bit added too (the lazy build has PIE only) and the environment equivalent, which is a different mechanism entirely: LD_BIND_NOW=1 the loader resolves everything regardless of what the file said

Which brings up the distinction the dlopen concept already introduced, at the loader level rather than the library level.DT_FLAGS is a property of the file and the loader obeys it; LD_BIND_NOW is a property of the environment and overrides it; RTLD_NOW is a property of one dlopen call. Three levels, one decision, and it is worth being able to say which is which.

Does It Actually Work?

The structural claim first, and it is the one that distinguishes this page from a restatement of the earlier course:

$ readelf -dW lazy_now | grep -E 'BIND_NOW|FLAGS'
  0x000000000000001e (FLAGS)      BIND_NOW
  0x000000006ffffffb (FLAGS_1)    Flags: NOW PIE
$ readelf -dW lazy | grep -E 'BIND_NOW|FLAGS'
  0x000000006ffffffb (FLAGS_1)    Flags: PIE

One bit added to DT_FLAGS, one to DT_FLAGS_1, and that is the entire difference in the dynamic section. Everything else about the file is the same, and the .plt is identical to the byte.

Now the question the earlier course could not ask, because it is about counting: how many binding decisions does the loader make?

$ for f in lazy lazy_now; do
    printf "  %-9s %s binding(s) for libfn\n" $f \
      "$(LD_DEBUG=bindings ./$f 2>&1 | grep -c libfn)"
  done
  lazy       1
  lazy_now   1
$ LD_BIND_NOW=1 LD_DEBUG=bindings ./lazy 2>&1 | grep -c libfn
  1

One, in all three configurations. This is the finding, and it is worth sitting with: lazy binding does not make more decisions and does not make a cheaper decision later. It makes the same single decision, at a different time. The program calls libfn a thousand times and the loader is involved exactly once in every configuration — at load time for two of them and at the first call for the third.

So what does lazy binding actually buy, and what does it cost? Both are narrow:

IT BUYS the loader does not have to resolve a symbol the program never calls. a library with 3000 exports linked into a program that calls 12 of them does 12 resolutions, not 3000. IT COSTS the first call to each function is a call through the resolver: push the link_map, push the symbol name, call, patch the slot, jump. a few hundred cycles, ONCE. IT ALSO .got.plt stays as a section, because its COSTS slots have two states. (measured: 0x20 bytes here, and the section is absent entirely under -z now)

And the honest limit: no timing claim. A two-million-call microbenchmark on this machine gave 5.1, 3.8 and 4.5 nanoseconds per call across lazy, -z now, and LD_BIND_NOW=1 on the lazy binary. That is noise, and this course does not teach it as a result. The structural facts above are exact and reproducible; the wall-clock difference on a modern machine for a single function call is not a claim anybody could defend. The Relocations course declined a similar claim for the same reason.

A Real Example

Where the choice actually matters, which is not where most people expect.

  lazy binding is a SECURITY decision more than a
  performance one, and the reason is historical.

  the resolver runs arbitrary code at the moment of
  the first call, from inside what looks like an
  ordinary call instruction. a process that has been
  checked by a sandbox and an audit tool has not yet
  run any of that code when the check happens.

  so: -z now is what makes a binary's entire dynamic
  behaviour happen at a moment you can name -- exec, or
  the first instruction -- rather than at a moment
  determined by which branches the program happens to
  take.

  what it costs: a slow start proportional to the
  number of PLT entries, and a failure at startup
  rather than at first call, which for a library with
  an optional dependency is a real behaviour change:

    a plugin that is PRESENT   -> works either way
    a plugin that is MISSING  -> -z now: the program
                                refuses to start
                              -> lazy: the program
                                runs, and dies if it
                                ever calls in

That second case is the one to hold on to, and it is a genuine trade rather than a “security is always better”. A program that optionally uses a library should be lazy, because eager binding turns a missing optional feature into a fatal startup error. A program whose entire job is to be auditable should be eager, because the audit is worthless if the interesting code runs later.

And the modern development, which is a good illustration of how this area has moved: lazy binding is losing ground because RELRO and -z now together are better, and the combination is now the default on most distributions.

  $ readelf -lW lazy_now | grep -E 'GNU_RELRO|GNU_STACK'
    GNU_RELRO  0x0000000000003db8 ... R      0x1
    GNU_STACK  0x0000000000000000 ... RW     0x10

  $ readelf -lW lazy | grep -E 'GNU_RELRO|GNU_STACK'
    GNU_RELRO  0x0000000000003db8 ... R      0x1
    GNU_STACK  0x0000000000000000 ... RW     0x10

  the relocations are RELATIVE either way -- they are
  resolved by the loader at startup REGARDLESS of
  -z now, because there is no PLT involved. -z now
  only changes the PLT ones, and those are the ones
  that stay writable.

Which is the important structural point and the reason -z now helps hardening at all.R_X86_64_RELATIVE — the relocation for every absolute pointer in the program — is applied at load time either way, and the page containing those slots can therefore be made read-only afterwards. That is GNU_RELRO. The PLT slots are the exception, because under lazy binding they are still holding resolver addresses and must stay writable. Close that hole and you have to have resolved them already, which is exactly what -z now does. The two flags are the same decision seen from a performance angle and a security angle.

Try It Yourself

$ cd courses/dyn/assets/samples
$ ./build_samples.sh 2>&2 | sed -n '/D7/,/D8/p'
$ python3 crosscheck.py 2>&2 | sed -n '/\\[7\\]/,/\\[8\\]/p'

Then find the boundary of the claim, which is a better exercise than repeating it:

  1. Count the bindings in a program that calls
     libfn ZERO times. Now one that calls it once.
     Now once a million times. Plot the three. (This
     is what "one decision" means, and the numbers are
     0, 1, 1.)

  2. Build a .so with 3000 exported symbols and a
     program that calls two of them. How many bindings?
     (Two, under lazy. Under -z now, 3000.) That is the
     real argument for lazy binding and it is worth
     seeing the number.

  3. Now add -Wl,-z,now and look at .got.plt again for
     the 3000-symbol library. Does the section come
     back? (It should not -- the slots still have one
     state each, just a different one.)

  4. With a shared library present, compare
     readelf -lW on the lazy and -z now builds and
     check GNU_RELRO's p_memsz. Does it change? (It
     should NOT, because RELATIVE relocations are
     resolved either way. Understanding why is the
     exercise.)

  5. Take an optional plugin: load it with dlopen
     RTLD_LAZY, call a function, and let it be missing.
     Then with RTLD_NOW. (The second fails at load, the
     first fails at the call. Which behaviour does a
     program want? It depends entirely on whether the
     plugin is a dependency or a feature -- and working
     out which is the design question.)

Exercise 2 is the one that gives lazy binding a real justification, and it is the argument its proponents actually make. Under -z now the loader resolves all 3000 exported symbols; under lazy it resolves the two that are called. That is a 1500× difference in loader work, and on a system with hundreds of libraries it is the difference between a fast start and a slow one. The cost is the opposite for a library with few exports and many calls, which is why the honest answer is “it depends on the shape of the program”.

Exercise 4 is the one that connects this concept to the security thread, and the surprising answer is that GNU_RELRO’s size does not change. Because R_X86_64_RELATIVE is resolved at load time either way. The reloc course measured that relocation type as the loader’s most common job, and this is where its necessity becomes obvious: a program full of absolute pointers has hundreds of them, and the only reason the page can be read-only afterwards is that the loader finished with it before main. Eager binding is not about performance; it is about closing the one gap that RELRO cannot close on its own.

Connect

This is the timing half of module 3, and it is the smallest correction in the course — which is itself worth noting. The symbol-resolution course said “-z now changes no code” and proved it. That is true and it is incomplete, and this concept supplies the missing half: .got.plt disappears and .got grows to absorb it. A course that stopped at “no code” would have left the reader thinking nothing at all happens, which is the wrong mental model for a flag whose purpose is security.

The connection to the Relocations course’s four-group model is the mechanism. That concept put JUMP_SLOT in the loader’s group — group 4, the relocations that exist so something can be decided later — and this concept is the measurement of how much later. A JUMP_SLOT is a number the loader writes into a slot; -z now changes only when. The relocation type is identical, the count of decisions is identical, and the section layout changes anyway — which is the clearest demonstration in the course that a relocation record describes a computation and not a moment.

The second connection is to the dlopen concept, and the three-level distinction it set up. RTLD_LAZY/RTLD_NOW at dlopen, LD_BIND_NOW in the environment, DT_FLAGS in the file. Three levels of the same decision, and the precedence runs from weakest to strongest. A program linked with -z now still gets lazy behaviour for a library it dlopen’s with RTLD_LAZY, because the file’s flag applies to the file’s own PLT and the call-site flag applies to the call. That is worth being able to state, and it is exactly the kind of thing that becomes obvious once you have the two concepts side by side.

And the connection forward, which completes the module, is the one place the loader does something no linker could. Where the Thread Blocks Come From is about storage: the Relocations course established that a thread-local variable has no address until a thread exists, and that general-dynamic TLS is the one relocation resolved by calling a function. Resolving the symbol is only half of it; something also has to allocate the block, and the loader is doing that with a static-surplus-versus-dynamic decision that has no link-time analogue at all.

One connection outward, for the security framing that runs through the course. The PIE concept listed the attack classes PIE removes and the ones it does not, and named a vtable or function pointer in read-only data as needing RELRO rather than ASLR. This concept is where RELRO gets its teeth: the reason the GOT can be read-only is that its only unresolved occupant is the RELATIVE slots, and the reason that is true is that the loader finished with them before main. -z now is not a separate hardening measure bolted on afterwards — it is what makes the existing one complete.

render_lesson_js(&mut page)