Keyboard Shortcuts

Ctrl+K
Open search
Esc
Close search / dialog
↑
Back to top

Callee-Saved and Caller-Saved, by Experiment

24 minadvancedModule 1: The Frame: What a Call Owes WhomContinues from x86-frame
24 min · The x86-64 ABI

Why this matters

Sixteen general registers, and a rule that splits them nine and six with one left over. The rule exists for a reason that follows directly from the previous concept: a caller cannot keep anything live in a caller-saved register across a call, because the callee is entitled to overwrite it. So anything the caller wants to keep has to live somewhere that survives, and on x86-64 the places that survive are six registers and the stack.

That is the whole reason the set is the size it is. It is not a design preference. It is the arithmetic of a machine with 16 registers where six are already spoken for by arguments and one more by the return value, leaving exactly the nine that the split describes.

But “the six callee-saved registers are rbx, rbp, r12–r15” is a sentence most people have read and none have checked, and the checking is worth doing, because the table is not the claim. The claim is a behaviour: put a value in %r12, call something, and see whether it is still there. That is a five-line experiment, and the next section runs it.

The model: the rule, and why a leaf gets an exemption

Here is the whole convention, and it is a rule about who pays:

CALLEE-SAVED: rbx rbp r12 r13 r14 r15 CALLER-SAVED: rax rcx rdx rsi rdi r8 r9 r10 r11 NOT IN THE TABLE AT ALL: rsp, and rip A callee may write any caller-saved register freely and must write nothing back. A callee that writes a callee-saved register must save the CALLER's value first and put it back before returning: pushq %rbx movabsq $0x9999999999999999, %rbx addq %rbx, %rdi popq %rbx Six bytes. That is the entire rule.

Two things in that table are worth stopping on, and neither is usually mentioned.

%rsp is not callee-saved. It is not in the table because it is not a register the convention allocates: it is the one register whose value is the ABI's own state, and a function that changed it without changing it back would not return. So the nine-and-six split is not “the sixteen registers, less one”; it is fifteen registers, and the split is nine caller-saved and six callee-saved. That is why the arithmetic in the previous section comes out at nine.

%rbp is a callee-saved general register and nothing else. There is no frame descriptor on this architecture, and push %rbp; mov %rsp,%rbp is a compiler convention. The direct evidence is that gcc -O2 uses %rbp as ordinary scratch, and the corpus this course audits shows it doing exactly that: mx at -O2pushes %rbp and then moves %rcx into it — the third of its four integer arguments — because %rbp is callee-saved and the value in it belongs to the caller.

And the leaf exemption is the interesting part, because it is not in most tables. A leaf function may clobber a callee-saved register without saving it, provided the compiler knows no caller had a live value there. That is a fact about the callers, not about the function, and it is why a mechanical checker cannot settle the question — which is the subject of the last section.

The reality: the same register, two callees

Two callees. The first tramples every callee-saved register it is allowed to and touches nothing else. The second tramples all nine caller-saved registers, uses %rbx, and saves it. The caller puts a distinct marker in %r12, calls, and reads %r12 back:

$ ./abidump | sed -n '/CALLEE-SAVED, BY EXPERIMENT/,/^6B\./p'
  ARM 1  a callee that tramples rbx r12 r13 r14 r15 and nothing
         else.  The caller put 0x0202020202020202 in %r12.
         the caller's %r12 after the call = 0x2222222222222222   CLOBBERED
         and the callee put 0x2222222222222222 there instead.
  ARM 2  a CONFORMING callee that tramples all nine caller-saved
         registers and uses %rbx.  The caller put 0x0b0b0b0b0b0b0b
         in %r12; after the call %r12 = 0x0b0b0b0b0b0b0b   SURVIVED
  ARM 3  a conforming callee that USES %rbx: pushq, movq, addq, popq
         0x100 + 0x9999999999999999 = 0x9999999999999a99   EXACT
  ARM 4  the same for %r12                             = 0x9999999999999a99   EXACT
  ARM 5  THE DIRECTION FLAG.  %rflags inside a conforming callee
         0x0000000000000000, and bit 10 (DF) = 0
         %rflags left behind by a NON-conforming callee
         0x0000000000000646, and bit 10 (DF) = 1
         The ABI makes DF the CALLER's problem: it must be 0 on
         entry and on exit to any function.  Which is why this
         file CLEARS it again on the very next line.
         DF after the `cld`                0x0000000000000246, bit 10 = 0
  ARM 6  %rbp, the sixth callee-saved register, which is a special
         case: a caller built with a frame pointer ends in `leave`,
         which is `movq %rbp,%rsp; popq %rbp`.
         a callee that clobbered %rbp     -> KILLED by a signal 7  (7 is SIGBUS)
         a conforming callee              -> returned normally
  

That is the table, established by behaviour. ARM 1 and ARM 2 are the same register, the same caller and the same call instruction, and the only difference is whether the callee pushed.

Arm 5 is a different rule, in the opposite direction

The direction flag is not a register and its rule is not the register rule. The ABI requires DF to be 0 on entry to and on exit from any function — and that is a rule the caller keeps, not the callee. A conforming callee clears it on the way out (ARM 5, first line: bit 10 is 0 inside it). A non-conforming one need not, and then the caller finds bit 10 set in its own %rflags, which is what the second line shows. A string instruction in the middle of the caller's loop will then walk backwards through memory instead of forwards, and nothing will say so.

And this course learned that the hard way, which is why the artifact clears it on the line after measuring it. The first version of abidump called the non-conforming callee, read the flag, printed the number, and moved on. Then the artifact hung. Not crashed — hung: twenty-five minutes of user time, no signal, no diagnostic, no output, in a program whose every other failure mode in this file is loud. The non-conforming callee had set DF in the artifact's own process, and with DF set every rep-based string routine in the C library scans backwards. The first strstr the artifact made after that point never returned. The whole run now takes 1.4 seconds instead of timing out, and the difference between those two facts is one cld and the awareness that the caller in ARM 5 was this program.

Notice what makes it a nasty bug rather than an interesting one: there is no compiler diagnostic for a callee that leaves DF set, because DF is not in any register and there is no callee-saved slot it could have been preserved in. The ABI's remedy is a sentence in a document. And a hang is the worst possible symptom, because a program that stops with no output looks like a slow machine, not like a broken contract.

Arm 6 is why %rbp is worth its own sentence

Clobbering %rbx loses a value. Clobbering %rbp kills the caller, because a caller built with a frame pointer ends in leave, which is movq %rbp,%rsp; popq %rbp — so a corrupted %rbp both restores the stack pointer to nonsense and returns to the word at that nonsense address. The measurement is a signal: SIGBUS, which is what a stack access that runs off the end of the mapping gives you, with the conforming callee as the control that returns normally. Note that the caller's side of this arm has to be written in assembly, because gcc refuses to let an inline asm block name %rbp at all.

And the two floating-point control registers point the other way

The one place the direction reverses, and it is worth a table of its own because nothing else in the ABI does this:

$ ./abidump | sed -n '/6B\./,/^  Note the field/p'
6B. WHERE THE DIRECTION IS REVERSED, AND IT IS MEASURED.

      MXCSR before the call   0x00001fa0  rounding control 0
      MXCSR after  the call   0x00006000  rounding control 3
      MXCSR after  a clean one 0x00006000  rounding control 3
      x87 control word before 0x037f  PC 63, RC 0
      x87 control word after  0x0250  PC 16, RC 0

      MXCSR after the CALLER restored it  0x00001fa0  rounding control 0
  

A callee changed the rounding mode from nearest to toward zero and the precision control from extended to double, and put neither back. MXCSR's control bits are caller-saved: a callee may change them freely. The x87 control word goes the other way and is callee-saved, and here the same callee changed it without restoring it either — the artifact's callee is deliberately non-conforming on both, so the two rows are a controlled comparison rather than two accidents.

The consequence is the reason this is on a page about a contract. A function that rounds differently from its caller and does not say so is a wrong answer, not a slow one, and there is no diagnostic: the arithmetic is perfectly well-defined, just not the arithmetic you asked for. And the cost of the reversal is asymmetric. The compiler pays almost nothing to honour it, because it is three instructions in the prologue. The programmer pays when a function nobody wrote changes the mode and the next value is compared against a tolerance instead of being expected to be exact.

Worked: an ABI linter, and the zero that was hollow

A convention is a contract between two compilers, so a contract between two compilers is the one kind of thing you can check mechanically. The artifact ships a checker that reads a real object file and asks, for every function: does this write a callee-saved register without pushing it first?

Run over glibc — the static library, because the check needs each function's extent and a stripped shared library has no symbol sizes:

$ ./abidump | sed -n '/^  LIBRARY |/,/^  UNMODELLED/p'
  LIBRARY | libc.a | 6238972 bytes
  FUNCTIONS | 4561 functions, and the register census is:
  CENSUS | rbx | 1670 | 1598
  CENSUS | rbp | 2555 | 2503
  CENSUS | r12 | 1242 | 1193
  CENSUS | r13 | 981 | 945
  CENSUS | r14 | 885 | 850
  CENSUS | r15 | 627 | 600
  CLOBBERFLAGS | 271 | 94 functions
  CLOBBERKIND | leaf | 26 | nonleaf | 245
  CLAPPERCONTEXT | 97 | 174
  CLAPPERBOTH | 3 | 151 unclassified after both exemptions
  

Read the census column first, because it is the one that makes the flags column mean anything. 1,670 functions write%rbx; 1,598 of them save it first. The checker looked at 1,670 real cases and is silent on 1,598 of them, which is a very different sentence from “the checker found nothing.”

Read the kind column next, because it is where the interesting result is. 271 flags across 94 functions, and 26 of them are in leaves — the exemption from the previous section, 26 real instances of it in a real library. And 97 of the 271 are in functions whose entire contract is to install a previously saved register context: longjmp, backtrace, swapcontext. Those functions are required to destroy the callee-saved registers, because restoring a context means writing them. A mechanical rule cannot know that.

And the two exemptions overlap, which is the arithmetic lesson hiding inside the table. A function can be a leaf and a context restorer — setjmp and its relatives are exactly that — so subtracting one count from the other is only valid if they are disjoint, and nothing in the ABI says they are. Three of the 271 are in both, so the remainder is 151 and not the 148 this page originally printed. The first draft subtracted anyway, got a plausible number, and was wrong by three out of 271 in a way no reader could have seen. The linter now reports the overlap as its own row, and the artifact's tenth retraction is about the arithmetic rather than about the code.

WHAT THE FLAGS ARE, AND WHAT THEY ARE NOT 271 register uses in 94 functions, and: 26 in leaf functions, which may do it if the compiler knows no caller had a live value there -- a fact about the CALLERS, not about this function 97 in functions whose contract is to restore a saved context, which must do it 3 in BOTH, because setjmp and its relatives are leaves as well as context restorers 271 - 26 - 97 = 148, and 148 is WRONG, because the two exemptions were counted independently and 3 functions are in both. The true remainder is 151. Those 151 are unclassified by a rule that cannot see intent, and a reader who reports them as bugs is reporting the rule, not the library.

And the first version of this linter reported nothing at all

This is the part worth keeping, and it is the reason the course ships a control file. The first version of abilint.py ran over libc, libm, the dynamic loader, libcrypto and /bin/ls — 9,255 functions — and reported zero violations in every one of them. Every one of those zeroes was hollow. The function that normalises a register name had its ternary the wrong way round:

THE BUG, IN ONE EXPRESSION return ("e" if m.group(1) == "r" else "r") + m.group(2) For the operand "r12" that returns "e12". Nothing was ever recognised as a callee-saved register, so the clobber check compared nothing against anything, and reported nothing found. 9,255 functions, zero findings, and the zero was a fact about the CHECKER and not about glibc. THE FIX WAS NOT TO BE CAREFUL. It was to build control.s, a file whose every function is a violation on purpose, and require the linter to flag all of them AND NOT to flag the two conforming ones.

The control is in the artifact's own output, and it is the number that makes the rest of the section checkable:

$ ./abidump | sed -n '/THE POSITIVE CONTROL/,/flags everything/p'
  THE POSITIVE CONTROL, without which every zero above is hollow:

    FLAGGED   ct_clobber_r12
    FLAGGED   ct_clobber_r13
    FLAGGED   ct_clobber_rbx
    CLOBBERFLAGS | 3 | 3 functions
    ALIGNCALLS | 5 | 1   <- and the one intended alignment violation
    ct_ok_leaf and ct_ok_leaf4 are conforming and are NOT in that
    list, which is the other half of the control: a linter that
    flags everything is caught as loudly as one that flags nothing.
  

Three flags, all three intended, and the two conforming functions absent. A checker that examines nothing and a checker that finds nothing print the same line, and only a positive control tells them apart. That is the seventh time in this collection that the same lesson has cost something, and it is now a standing part of every harness in it.

Apply it

  1. Break the control and watch the checker find it. Delete the pushq %rbx / popq %rbx pair from ct_ok_leaf in control.s and re-run. (Expect group H2 of the harness to fail on the check that names ct_ok_leaf specifically, and the libc flags to go up by one. That is the completion criterion for the course, and it takes one line to do.)
  2. Add a leaf clobber to your own assembly and see whether it works. Write a leaf that uses %r12 without saving it, link it, call it from C. (Expect it to work, every time, and that is the correct result and the wrong thing to rely on. It works because the compiler can see your C caller and knows it has nothing live in %r12 across the call. Change the caller — put a loop counter in %r12 by hand in inline asm, or call the leaf from a function that keeps something there — and expect a wrong answer rather than a crash, because the symptom of a clobbered callee-saved register is somebody else's value.)
  3. Set the direction flag in a callee and watch the caller.(Expect the ARM 5 result: bit 10 clear inside a conforming callee, set after a non-conforming one. Then write a loop that copies a string forwards with rep movsb, call the dirty callee, and confirm the copy runs backwards. Nothing faults. Nothing warns. The array is simply written in the other order, and the failure is a data-corruption bug that looks like a logic bug.)
  4. Change the rounding mode in a callee. Use the artifact's abi_fp_dirty as a model, or write ldmxcsr yourself. (Expect the caller's arithmetic to round toward zero afterwards, and expect the caller to be able to fix it in one instruction — which is the point. MXCSR's control bits are caller-saved precisely so that a function doing heavy floating-point work can set the mode once instead of restoring it on every return. The cost is that every other function doing floating-point work has to save it too, and the convention says so in a table most people have never read.)
  5. Run the linter over your own project's assembly, and then over its compiler output.(Expect the two to come out differently, and expect the difference to be the leaf exemption: a compiler that knows its callers will use registers freely in leaves, and a hand-written function that cannot know anything will save everything. Both are legal. A project with a callee-saved register clobbered in a hand-written leaf that is called from somewhere with a live value is a bug, and the linter will not tell you so — which is why the last concept is about what a checker can and cannot establish.)

Connect

Backwards, the stack frame is the argument this concept's whole first half rests on: a caller cannot keep a value in a caller-saved register across a call, and the red zone is not available to a function that calls anything, so the stack is what is left. The multiprocessor course's sharing concept is where “save it somewhere that survives” stops being about registers and starts being about which memory is visible to which core, and the callee-saved convention is the register-level shadow of that problem.

Forwards, one practical consequence. The linter in this concept is a checker, and a checker needs a spec to check against — which for x86-64 means the CFI that describes a frame to something that was not there when the frame was built. The unwinding concept is about that second description, and it is where a frame pointer stops being a debugging convenience and becomes the only thing a profiler has when a function has no table.

Outward, the security course owns a claim that is adjacent and different. The stack-protector concept is about protecting the return address, which this course has just established lives at 0(%rsp) at a callee's first instruction — the same eight bytes as the first slot of the red zone. That is not a coincidence to exploit; it is a coincidence to notice, and it is the reason a canary sits between the saved frame pointer and the return address rather than in front of the locals. The two concepts are worth reading together precisely because the same address has two owners depending on whether a call happens.

render_lesson_js(&mut page)