Keyboard Shortcuts

Ctrl+K
Open search
Esc
Close search / dialog
↑
Back to top

Four Ways In, and One of Them Is Not a Door

27 minadvancedModule 2: Getting InContinues from priv-vectors
27 min · Exceptions, Privilege and Mode Changes

Why this matters

There is a gate table, a DPL on every gate, and a rule that refuses a transition when the DPL is too low. It is very tempting to conclude that every transition into the kernel goes through that check. It does not. Two of the three instructions people use to enter the kernel never look at the table at all, and the third is a vendor disagreement rather than a mechanism.

That is worth understanding for its own sake, because “the CPU checks the DPL” is the mental model that a hundred explanations of system calls share, and it is wrong in a way that matters. It matters for a compiler author, who needs to know which entry sequences are checked and which are not. It matters for a kernel author, who has to do by hand what the hardware will not do. And it matters for anyone who has assumed that “the hardware enforces privilege” is a single uniform mechanism — because on this particular machine, the most popular fast entry path is a documented divergence between two vendors rather than a mechanism at all.

The model: what each instruction actually does

There is a family of x86 instructions for entering the kernel, and they are not variations on a theme. They differ in where the target comes from and what checks happen on the way.

INT n / INT3 where from? the IDT, at offset n*16 checked? YES -- gate.DPL < CPL -> #GP saves? CS, RIP, RFLAGS, and an error code cost a full trap: table lookup, DPL check, present check, canonicality check SYSCALL where from? IA32_LSTAR, a model-specific register checked? NO. There is no gate, so there is nothing whose DPL could be checked. saves? RCX (the return RIP) and R11 (RFLAGS) cost read three MSRs, load CS and SS, done SYSENTER where from? IA32_SYSENTER_EIP checked? NO, on either vendor valid? Intel: yes. AMD: NO -- illegal in long mode, raises #UD cost the cheapest of the three, where it exists The vDSO where from? no transition at all checked? nothing to check -- no mode change occurs cost a normal function call into a page the kernel mapped into this process

The SYSCALL row is the important one, and it is the one that overturns the mental model. Read Intel's own pseudocode for the instruction: its entire guard clause is

IF (CS.L != 1) OR (IA32_EFER.LMA != 1) OR (IA32_EFER.SCE != 1) THEN #UD; FI; ... CPL := 0;

There is no CPL test in there.SYSCALL is not a privileged instruction. It does not check who you are; it unconditionally sets the current privilege level to zero, reads three MSRs for the target and the new segment selectors, and goes. Every argument the program supplies is then the kernel's problem to validate, and the kernel must validate all of it, because the hardware did not.

Compare that with INT n, which does the DPL check in hardware and therefore can trust that only callers the kernel intended got in. Trade one CPU cycle at the entry against auditing every argument after it. That is the design decision, and it explains both the speed and the security model.

The reality: four arms, one measurement, and one refusal

Five timing arms, interleaved, minimum of nine attempts each, in TSC ticks. Remember rule 1 of the artifact: the TSC is a fixed-rate reference clock, not a count of core cycles, and the core clock moves inside a run. Only the ratios mean anything.

$ ./privbench | sed -n '/SYSCALL, getpid/,/vdso-clock/p'
      SYSCALL, getpid               1238.5
      int $0x80, getpid             6095.6
      libc getpid()                 1236.5

      SYSCALL, clock_gettime        1668.8
      vDSO  clock_gettime             60.4

      int80 / syscall                 4.92x
      syscall-clock / vdso-clock     27.65x

The second pair is the result

clock_gettime twice. Same function, same argument, same result, same process, same binary, same moment. The only difference is whether the CPU changed privilege level on the way. That 27× is the cost of a mode change, isolated. It is also the number the vDSO exists to eliminate, and it is why reading the clock in a well-written program costs tens of nanoseconds and in a naive one costs hundreds.

The first pair is a trap, and the artifact refuses to conclude

The obvious story about int $0x80 is that the Linux i386 ABI hands arguments to the handler through I/O ports, one port access per argument, and I/O is not something a fast path pipelines. It is a good story. It is also not a conclusion this measurement can support, and the artifact says so rather than printing the story:

$ ./privbench | sed -n '/What the first pair does NOT prove/,/section 8, R1/p'
   What the first pair does NOT prove, which this file is careful about:
   It is tempting to say `int $0x80 is slower because it does one I/O
   port access per argument`.  That is TRUE of the Linux i386 entry stub
   -- but the port accesses are executed by the KERNEL'S HANDLER, not by
   the INT instruction.  So this measurement cannot separate the cost of
   the hardware entry from the cost of two different kernel stubs.  The
   honest claim is the ordering and the ratio; the mechanism is named as a
   hypothesis and NOT as a finding.

This is worth more than the ratio it declines to explain. The first draft of this course tried to isolate the mechanism by timing the same syscall with zero arguments and with six. It cannot work, and the reason is instructive: the number of ports is decided by the kernel's knowledge of that syscall's declared arity, which a caller cannot vary at all. The experiment was removed rather than reported. An experiment that cannot answer its question is not data.

The third instruction: a divergence, not a bug

$ ./privbench | sed -n '/The third instruction/,/Both manuals are correct/p'
   The third instruction:
   sysenter                (the #UD)  SIGILL   si_code=2    (0x02)

   This is the course's sharpest portable result and it is NOT a bug.
   Intel SDM Vol 2B lists SYSENTER as VALID in 64-bit mode: "When
   executed in IA-32e mode, the SYSENTER instruction transitions the
   logical processor to 64-bit mode", with 64-bit-mode exceptions
   "same as in protected mode".  AMD64 APM Vol 2 sec 6.1.2 says the
   opposite: these instructions are ILLEGAL IN LONG MODE and result in
   an invalid opcode exception (#UD).  Both manuals are correct.  They
   disagree.

So a program that reaches for SYSENTER as its fast system-call path is correct on one vendor and takes SIGILL on the other, and there is no erratum to file against either company, because each is accurately documenting its own silicon. The Linux kernel encodes the split explicitly — arch/x86/kvm/emulate.c contains a comment that spells out both the 64-bit-mode and the compatibility-mode differences — and QEMU commit c046a42c does the same thing for the same reason. A build-time CPUID check is the only defence.

And one correction, because the author of this course got it wrong first. The folklore says “SYSENTER from ring 3 raises #GP(0).” That is false on both vendors. There is no CPL check anywhere in SYSENTER; it is unprivileged by design, and running it from ring 3 is its entire purpose. Intel's only #GP(0) condition is a null IA32_SYSENTER_CS. The #UD this machine raises has nothing to do with privilege at all — it is AMD having removed the instruction from long mode. Two wrong beliefs cancelling into a right observation is not evidence, and the draft that nearly published that story is retraction R2 in the artifact.

Worked: what each door costs you, concretely

DESIGN COST OF EACH ENTRY, which is not the performance cost and is much higher: SYSCALL / SYSENTER + no argument validation needed from the CPU + three registers are clobbered: RCX, R11, and the flags. You must tell your compiler. - EVERY argument must be validated by hand. The hardware checked nothing. A missed check is a kernel bug reachable from unprivileged code. - you must arrange the stack yourself. SYSCALL does NOT save RSP. (SYSEXIT is the privileged one and it is #GP(0) at CPL != 0, and SYSRET is #GP(0) at CPL != 0 -- so the return path has a check and the entry path does not, which is backwards from what you would guess.) INT n + the CPU checks DPL, present, and canonicality + the CPU pushes the whole frame including an error code - the frame layout is fixed by the architecture - vector numbers are a shared, scarce namespace THE vDSO + no mode change, so none of the above + the kernel can implement it however it likes - it is a copy in user memory. It can be OUTDATED relative to the kernel. A user program that trusts a vDSO result about something the kernel knows better is trusting a cache. - the kernel must keep its ABI stable forever, because user binaries outlive kernels. That is a much bigger constraint than a gate table.

The vDSO's cost deserves the emphasis, because it is the one nobody thinks about. Every operating system pays it and most people never see it: once the kernel publishes a vDSO with a set of exported functions, it can never change their behaviour in a way a running program would notice, and it can never remove one. That constraint outlives every kernel version decision, and it is why some of the fastest things in Linux have been awkward for twenty years.

Apply it

  1. Re-derive the 27× yourself. Time clock_gettime through syscall(SYS_clock_gettime, ...) and through the libc wrapper. Then time syscall(SYS_getpid) and getpid(). Compute the four ratios. Expect the two clock ratios to agree closely and the two pid ratios to not, and work out why before you read on.Which of the four is a measurement of the boundary, and which is a measurement of the boundary plus something else?
  2. Find out whether your machine has a vDSO and whether libc is using it.ldd --version names the mechanism; strace -c -e trace=clock_gettime shows whether the syscall happens at all. On glibc 2.31+ the vDSO is conditional at runtime, so the answer is a property of your kernel and not of your libc. Now explain why a program built on such a machine and run on one without it must still be correct.
  3. Write the argument validation SYSCALL obliges you to do. Pick one syscall and list every argument the kernel must check, and what it must check it against, before it may dereference it. (Pointer: range, alignment, and the object it names. A pointer the hardware will happily put in a register is a pointer the kernel has not yet agreed is real.)
  4. Decide what your own project does about SYSENTER. One of: use SYSCALL always; use SYSENTER behind a CPUID check; never enter the kernel at all. State which, and state what you will do when the check is wrong. The third option is not a joke — the vDSO is how the real answer looks.

Connect

Forward, the convention concept takes the row labelled “syscall number” in this model — the number SYSCALL does not put anywhere — and reads it out of the kernel's own machine code, because the architecture never mentions it. The canonical-address concept then finds the other thing that makes ring 3 ring 3.

Backwards, the dependency is direct. The vector concept established that a process names a vector and the kernel owns what happens, and this concept is the consequence: an instruction that bypasses the table gets its speed from bypassing the ownership too. The executable-images course's vDSO concept already said that entering a syscall is expensive to enter and left it there; that sentence is now a measurement.

Outward, the portability lesson is the one to carry. Two companies document the same instruction incompatibly, both correctly, and no erratum exists. That is not a curiosity about the 1990s; it is the normal condition of a specification written by vendors rather than by a standards body, and it is why the three-architecture concept exists at all — AArch64 has SVC and RISC-V has ECALL, and neither has this particular disagreement, but each has its own. Read the second vendor's manual before you rely on the first vendor's manual. That is the whole transferable rule.

render_lesson_js(&mut page)