Every previous course has consumed shared libraries. This one builds them, and the first thing to understand is that a shared library is a much narrower thing than it looks.
$ cat lib.c
static int static_helper(int x){ return x + 1; } /* NOT exported */
int lib_value(void){ return static_helper(0); } /* exported */
$ clang -fPIC -shared -o liba.so lib.c
$ nm -D --defined-only liba.so
0000000000001110 T lib_value
$ nm liba.so | awk '/ [tT] /{print " " $2, $3}'
T lib_value
t static_helper
One exported symbol out of two functions written. The upper-case T is .dynsym — the only table the loader searches. The lower-case t is in the full symbol table, where it is useful for debugging and invisible to every other process. A shared library is not a program; it is a program plus a short list of names it is offering to the world.
What -fPIC -shared actually does, as four separate decisions rather than one.
The second bullet is the one that surprises people.-fPIC and -shared are independent, and a file with only one of them is a common mistake in both directions:
So the two flags are not alternatives. Position independence is a property of the code; sharing is a property of the file. A shared library needs both, and each is answering a different question.
What the three names of a library are for, measured by building one with a version and a soname:
$ cp liba.so libfoo.so.1.2.3 $ ln -s libfoo.so.1.2.3 libfoo.so.1 $ ln -s libfoo.so.1 libfoo.so $ clang -fPIC -shared -Wl,-soname,libfoo.so.1 -o real.so lib.c $ readelf -dW real.so | grep SONAME 0x000000000000000e (SONAME) Library soname: [libfoo.so.1] $ clang -o useprog main.c -L. -l:libfoo.so.1 -Wl,-rpath,'$ORIGIN' $ readelf -dW useprog | grep NEEDED 0x0000000000000001 (NEEDED) Shared library: [libfoo.so.1]
Three names, three audiences, and the middle one is the contract.libfoo.so.1.2.3 is the file — it changes whenever the code changes. libfoo.so.1 is what the program records, forever. libfoo.so is what a developer types on a link line. Upgrading 1.2.3 to 1.4.0 changes the first and the third and must not change the second, or every program on the system stops running.
And here is the demonstration that -soname is not decoration. The libraries in this course’s sample directory were built without it:
$ readelf -dW liba.so | grep -c SONAME 0 $ readelf -dW prog | grep NEEDED (NEEDED) Shared library: [libb.so] (NEEDED) Shared library: [liba.so]
No soname, so the program recorded the file name. Rename liba.so to liba2.so and prog stops finding it — not because anything is broken, but because the string in .dynamic no longer names a file. That is the entire reason -soname exists, demonstrated by its absence.
Now the parts of the file, and which of them other processes can see:
$ readelf -SW liba.so | awk '$2 ~ /^\./ {printf " %-18s %6s %s\n", $2, $6, $3}'
.text 000017 AX
.dynsym 000060 WA
.dynstr 00003c WA
.gnu.hash 00001c WA
.rela.dyn 000048 WA
.rela.plt 000018 WA
.init 000013 AX
.fini 00000d AX
.init_array 000008 WA
.dynamic 0000e0 WA
.got 000010 WA
.got.plt 000010 WA
.data 000010 WA
.bss 000008 WA
Two of those are the interface and the rest is the machinery..dynsym plus .dynstr plus .gnu.hash is everything another process can learn about this library. .symtab and .strtab are present too, and nobody outside the link reads them — which is why strip can remove them and why the symbol-resolution course could measure .symtab growing to 0xc210 bytes in a static binary without anything being able to use it.
The three-name scheme is not a convention; it is what makes upgrades possible, and the failure mode when a library gets it wrong is a system that will not boot.
libfoo.so.1.2.3 the real file, replaced on upgrade
libfoo.so.1 a SYMLINK, never replaced
libfoo.so a SYMLINK, for the link line
$ ls -l libfoo*
lrwxrwxrwx libfoo.so -> libfoo.so.1
lrwxrwxrwx libfoo.so.1 -> libfoo.so.1.2.3
-rwxr-xr-x libfoo.so.1.2.3
install a new libfoo.so.1.3.0:
cp the new file in
repoint libfoo.so.1 at it
repoint libfoo.so at libfoo.so.1
and every program linked against libfoo.so.1 gets
the new code with NO relinking and NO change to its
own DT_NEEDED.
The middle symlink is the ABI contract and the only one that must be stable. Which is why “bump the soname” is a deliberate, disruptive act: a library that changes an interface must change the soname, and every program using it must be relinked. A patch release changes only the first name; a soname change is how you say “these are not compatible” in a way the loader can enforce.
And the practical cost of a .so that is a shared object in name only — the mistake of shipping a PIC executable renamed:
the giveaway, and it is one command:
$ readelf -dW libfoo.so | grep -c NEEDED
0 <-- it depends on NOTHING. so it provides
nothing to anyone who needs it, and
every symbol in it is undefined.
a real library almost always needs libc, and DT_NEEDED
is how you check whether the thing you built is a
library or an executable that got the wrong -o flag.
Which is the cheapest sanity check in the subject, and worth knowing because the failure is otherwise baffling: a program links against the “library”, reports every symbol undefined, and there is nothing in the error message to suggest the file is not what you think it is. The artifact uses exactly this: it refuses to treat an object with no .dynsym as a scope entry that can satisfy anything, and that is the same fact.
$ cd courses/dyn/assets/samples $ ./build_samples.sh 2>&2 | sed -n '/D1/,/D2/p' $ python3 crosscheck.py 2>&2 | sed -n '/\\[1\\]/,/\\[2\\]/p' $ python3 dynscope.py exports liba.so
Then build the three-name scheme yourself, because it is four commands and it is what every packaged library on your system looks like:
1. Build a library with and without -soname. Link a
program against each, and diff their DT_NEEDED.
(One records libfoo.so.1, the other records
libfoo.so.1.2.3. Then rename the file and see
which program still runs.)
2. Build with -fPIC but WITHOUT -shared, and with
-shared but WITHOUT -fPIC. What does the linker
say in each case, and is it the same diagnostic
the reloc course recorded?
3. Take liba.so and strip it: objcopy --strip-all.
Does it still work? Which sections disappeared, and
which of them was the interface?
4. Build a library that references a symbol nobody
defines. Does -shared accept it? (It must -- that
is the point. When is the error raised instead,
and by whom?)
5. Count: how many sections does a trivial .so have,
and how many of them can another process learn
anything from? (Three: .dynsym, .dynstr, and the
hash table that indexes them.)
Exercise 3 is the one that makes the interface/mechanism distinction permanent. Stripping a library removes .symtab and leaves .dynsym, and the program that links against it does not notice, because it never looked at .symtab. The two tables have the same symbols in them and completely different audiences, and stripping is the cleanest demonstration that they really are different things rather than one table shown twice.
Exercise 4 is the one that connects back to the whole course. A shared library with an unresolvable reference links cleanly — that is the definition of sharing — and the error is deferred to load time, when the scope search fails. Compare with a static link, where the same reference is an immediate error. Sharing moves the failure, and the next two modules are about what happens when it moves.
This opens the building module, and it is the mirror image of the ELF course’s shared-libraries concept in a way worth being explicit about. That course covered the three names and the commands that build one, and it was right to. What it could not show is what a shared library is, because every number in it came from a library someone else had built. This concept builds one and then looks at it: one exported symbol out of two functions, no SONAME because nobody set one, and a DT_NEEDED that records file names as a direct consequence.
The connection to the Relocations course’s cost measurement is the reason -fPIC is mandatory here rather than a preference. That concept measured the exact price: 17 instructions against 9 for eight global reads, 10 bytes against 6 per reference, and a dependent load that cannot start until the address load retires. A shared library pays that cost on every global reference it makes, forever, and the alternative is not available — a non-PIC shared object is rejected by the linker with a diagnostic naming the relocation type.
The second connection is to Interposition, and it is the reason this concept exists in this position. Interposition works because the loader searches .dynsym. So the contents of .dynsym are not a packaging detail; they are the set of candidates for every search the loader will ever perform. A symbol in that list is reachable by any process on the system that loads this library; a symbol outside it is unreachable to everything except the debugger. This concept establishes the list, and the next one is about controlling it.
Forward within the module, the two remaining concepts split the question that “what is in .dynsym” raises into its two halves, and keeping them apart is the point of having both. Hiding Things, and the Trap is about the export side: which names go in, and the flag that removes the ones you meant to keep. One Instruction Called -Bsymbolic is about the binding side: a name that is exported, and a call to it that should not be interposable. One is about who can see you; the other is about whether you can be seen.
And the connection forward into the run-time module is the failure this build deliberately permits. A shared library with an undefined reference is legal, which means the error is deferred to load time, and the mechanism that produces the error is the scope search. dlopen, dlsym, and the Mode Bits is the point at which the list grows at run time rather than at link time, and it is also the point at which a library can be loaded that the program never mentioned — which is a capability, an attack surface, and a debugging problem, in that order.