The last concept ended with a byte that is a prefix in one mode and an instruction in another. That only works because the one-byte map was already full enough that its spare values had to be reused. So the interesting question is not “what does 0x89 mean” — it is what shape is the map, and what does it do with the values it has no room for.
$ ./build_samples.sh 2>&1 | sed -n '/I5/,/I6/p'
the 256 one-byte values:
an instruction 228
legacy prefix 27
nothing 1
Over a tenth of the one-byte map is not an instruction at all. Sixteen of the 27 prefixes are the REX bytes from the last concept; the other eleven are the ones that predate 64-bit mode. That is not a design flaw, it is the arithmetic of a design that wanted room to grow in a space that could not grow.
And the escape has a hole in it on purpose, which is the single most useful fact in this concept and the reason the hardening course can end a bad indirect branch immediately rather than somewhere random:
$ ./build_samples.sh 2>&1 | sed -n '/escape and the holes/,/^$/p'
the escape and the holes in it:
0f 05 syscall -> syscall
0f 0b ud2 -- deliberately UNDEFINED -> ud2
0f 0e femms (3DNow!) -> femms
0f 1e fa bare: a multi-byte NOP -> nop edx
f3 0f 1e fa with F3: endbr64 -> endbr64
cd 80 the older syscall path -> int 0x80
0F 0B is UD2 — undefined instruction. It does nothing except fault. And the reason it exists is exactly the reason the last concept needed a chain check: an invalid indirect call should stop at the first instruction it reaches, not wander through memory executing whatever bytes are there. A deliberately-undefined opcode is a way of saying “if you are executing this, something has already gone wrong, and the polite thing is to die now.”
There are three maps, not one, and they are reached in a fixed order:
That last line is a design decision with a measurable consequence, and it is worth dwelling on. The escape byte was placed at 0x0F, and REX took 0x40–0x4F. The two ranges do not overlap, so the escape is immune to the mode collision. System instructions are exactly the ones you least want to be mode-dependent: a program that makes a syscall should not have a different encoding depending on which mode it was compiled for. 0F 05 is 0F 05 everywhere.
Now the part that is genuinely strange, and it is the reason the map has the shape it does. Some of the one-byte values are not instructions and not prefixes either — they are escapes for other escapes.CD is INT imm8, the 1990s system-call path, and it is still there. C4 and C5 were LES and LDS — instructions so thoroughly unusable in a flat address space that Intel repurposed them as the VEX prefixes for 128-bit vector code. So:
That is the general shape of the whole map, and it is the answer to “why is the opcode table so weird”. An opcode is a slot, and slots outlive their contents. The REX collision and the EVEX collision are the same phenomenon, and so is C4.
The counts above came from asking the oracle about all 256 byte values individually, not from reading a manual. That distinction matters, because the manual is the authority and the oracle is what a tool actually does — and where they differ, you want to know.
$ python3 - <<'PY'
import subprocess, collections
def one(bs):
open('_p.bin','wb').write(bytes(bs))
o=subprocess.run(['objdump','-D','-b','binary','-m','i386:x86-64',
'-M','intel','_p.bin'],capture_output=True,text=True).stdout
r=[l.split('\t') for l in o.splitlines() if len(l.split('\t'))>=3]
return [x[2].strip() for x in r if x[0].strip().rstrip(':').strip()]
pref=('data','rep','lock','cs','ds','es','fs','gs','ss','addr','rex')
c=collections.Counter()
for b in range(256):
g=one(bytes([b]))
if not g: c['nothing']+=1
elif g[0].startswith(pref): c['legacy prefix']+=1
else: c['an instruction']+=1
for k,v in c.most_common(): print(' %-18s %3d' % (k,v))
PY
an instruction 228
legacy prefix 27
nothing 1
Two of those three numbers are worth pausing on. 228 instructions in 256 byte values is a map that is nearly full — and the 16 that are missing are the prefixes, which is why REX had to collide rather than expand. And the “legacy prefix” bucket of 27 is 16 REX plus 11 older ones: 66, 67, F0, F2, F3, and the six segment overrides 2E 36 3E 26 64 65. Six of the eleven exist to name a segment, which in a flat address space means the DS and ES and SS ones are now decorative.
Now the hole, because it is the load-bearing one. 0F 0B is UD2 and the hardening course already made the point from the other direction: CET makes an indirect branch to a bad target fault, and this is the classic belt-and-braces landing pad. But the older, simpler version of the same idea needs no hardware at all:
$ python3 x86dec.py 0f0b 0f05 cd80 0f 0b ud2 0f 05 syscall cd 80 int 0x80
Three two-byte instructions, and they tell the whole story of system calls on this platform. 0F 05 is the modern path, CD 80 is the 1990s path, and 0F 0B is the answer to “what if the CPU is not where you thought it was”. A compiler emits UD2 immediately after a noreturn call site, so that if control ever reaches there anyway — because a function that promised not to return did — the program stops at the very next instruction instead of executing the bytes that follow, which might be a jump table or a string constant.
And ENDBR64 belongs in this concept too, because its encoding explains its behaviour. F3 0F 1E FA is a mandatory-prefix escape: the F3 is not decoration, it is what selects this instruction from the 0F 1E group.
So an endbr64 is harmless on a CPU with no CET, where it decodes as a four-byte NOP with an odd prefix, and meaningful on one that has it. That is why the hardening course measured the instructions being present while the property note claimed nothing: the instructions and the enforcement are separate, and this is the encoding side of that separation.
There is a method in this course that turned out to matter more than any single fact, and it belongs here because the opcode table is where it pays.
The decoder needs a table saying, for each opcode, whether it takes a ModRM byte and whether an immediate follows. That table is ~360 entries. Writing it by reading a manual is slow; writing it by hand from memory is how you get 0F F4 labelled HLT when it is actually PMULUDQ. So the crosscheck builds a minimal instruction from every entry and compares the length against the oracle:
Every one of those four errors was invisible in isolation. Each produced a plausible-looking disassembly of a hand-picked example. They were only findable by exhausting the table, because a decoder has no other way to find out it is wrong — it has no internal contradiction to notice.
That is the argument for the method, and it generalises past this course. A lookup table has no self-check. A table that is 95% right produces output that is 95% right and gives you no way to tell which 5%. A crosscheck that walks the whole table converts “I think this is right” into a number, and a number either holds or does not.
One thing the table deliberately does not do, which is a decision rather than a gap:
So the decoder is deliberately asymmetric. Length and structure are the contract; naming is a convenience. That is the right way round for this course, because the question a linker asks is where the bytes are, never what they are called.
$ cd courses/isa/assets/samples $ ./build_samples.sh 2>&1 | sed -n '/I5/,/I6/p' $ python3 crosscheck.py 2>&1 | sed -n '/I5/,/I6/p' $ python3 crosscheck.py 2>&1 | sed -n '/I9/,$p'
Then find the slots that changed hands, which is the better exercise:
1. List the opcodes in this build's table that
are prefixes now and were instructions
once. The decoder's table has the answer;
the manual has the history.
grep -n "'c4'\|'c5'\|'cd'\|'62'" x86dec.py
(C4 and C5 are VEX -- they were LES and LDS.
CD is INT imm8 and still is. 62 is EVEX and
was BOUND. Ask what each replaced and why the
replacement was worth more than the thing it
replaced. VEX is the interesting case: a
prefix is 2 or 3 bytes, and its whole job is
to carry bits -- a 4th operand, a wider
register -- that the legacy opcode had no
room for. The encoding grew a prefix layer
rather than growing the opcode.)
2. Now the opposite direction: find a byte
sequence that is a valid instruction in 32-bit
and an INVALID one in 64-bit, using the mode
collision from the last concept as a lever.
40 89 e8 valid in both, 1 insn vs 2
62 ... EVEX, invalid to a reader
without AVX-512
(There are others. 0F 01 with certain ModRM
values, and the whole far-jump family. The
general question is: which 32-bit instructions
became impossible, and what replaced them?)
3. Take the UD2 argument seriously and find the
compiler's own use of it:
clang -O2 -S -o - t.c | grep -A2 -B2 'ud2'
Count how many UD2 instructions appear in a
modest C file, and check that each one follows
something marked noreturn. Then write the
three-instruction program that has no UD2 and
see what happens when it falls through:
int f(void) { return 1; }
int g(void) { int x = f(); if (x) return x; x = 0; }
(The point is that UD2 is not a security
feature bolted on. It is the compiler saying
"control cannot arrive here", and making that
claim ENFORCEABLE in one byte. Compare the
cost: 2 bytes against what an equivalent
runtime check would cost.)
4. Decode a real binary and count the distinct
opcodes it uses, then check how many your table
can name:
objdump -d /bin/ls | awk -F'\t' '{print $3}' \
| awk '{print $1}' | sort | uniq -c | sort -rn | head -30
(Expect the top of that list to be entirely
table entries, and the tail to include SSE
opcodes the decoder lengths but does not name.
That asymmetry is the design, not an oversight:
a real binary is mostly the operations the
compiler emits most often, and those are the
ones worth naming.)
5. Finally: write your own opcode table for a
DIFFERENT architecture -- AArch64 -- and
compare the two shapes. (AArch64 is fixed-width
32-bit, so the table is a flat 4-byte field with
no prefixes and no length calculation at all.
The interesting part is not that it is simpler.
It is that a fixed-width encoding makes the
question this course is about -- how many bytes
is this instruction -- trivially answered, and
x86's difficulty is the direct price of
compactness. Neither is wrong. They are
different answers to "what do you want the
encoding for?".)
Exercise 5 is the one that gives the course its shape, and it is worth doing because it inverts the usual reading. AArch64’s encoding is four bytes, always, and that is a feature — a decoder is a table lookup, instruction boundaries need no chaining, and nothing can desynchronise. x86’s is 1 to 15 bytes and that is also a feature — code density, which mattered when a 64K memory was the constraint. The two designs are the same trade in opposite directions, and this course only exists because x86 chose the end that makes decoding hard.
Exercise 3 is the one that changes how you read a compiler. UD2 after a noreturn call is the compiler making a claim enforceable in two bytes, and the claim is “control flow cannot arrive here”. Once you know to look for it you will see it in every non-trivial binary, and it reframes a stray 0F 0B from noise into a proof obligation the compiler discharged. That is the same move as CET’s endbr64 and the same move as a relocation’s zero addend: a value in the file that asserts something about control.
The direct technical debt this pays is to the DWARF standard-opcodes concept, which taught that DWARF numbers instructions using per-architecture tables because the encodings differ. That course could state the fact and had to stop there; this is the concept that shows what one of those tables contains, and why the DWARF spec needs 20 pages of them. A fixed-width architecture has a one-column table. x86 has a table with a length column, a ModRM column and a prefix column, and the crosscheck in module 4 exists because a table that shape is exactly the kind that can be 95% right.
The connection to the linker-script concept is about the map’s exhaustiveness. That course found the default script’s rule for .plt and could not say why the section was sometimes empty; the REX concept and the ModRM concept supply the answer for the other half of that puzzle — a .plt entry is one instruction, and whether the entry needs a disp32 or a rip+disp32 depends on which ModRM class the encoding used. Three courses, one mechanism.
Two connections outward, both about the same idea in a different register. The CET concept measured that endbr64 is present in binaries with protection disabled, and this concept supplies the encoding: F3 0F 1E FA, a reinterpretation of an existing opcode by prefix. The reason the instructions can be present while enforcement is not is that the instruction and the request are two different files — the opcode map and the property note — and this concept is the half that lives in the opcode map. And the DWARF standard-opcodes concept is the other reader of the same table, for a different purpose: the hardening course taught that a hardening feature is a compiler decision, a linker decision, or a driver decision, and this course is the reference for the first of those three layers at the level of individual bytes.
One limit that is also the course’s method, stated plainly. The opcode table in the artifact covers 364 audited entries for length and structure, and names a subset. It is not a complete disassembler and does not claim to be: where it has no name it prints the opcode in hex rather than guessing, because a guessed mnemonic is a claim nothing checked. And no number here was read from Intel’s documentation — everything was measured against a reader and, where the two disagreed, the disagreement is recorded in research.md under “Oracle limitations” rather than resolved in Intel’s favour by assertion. That is the collection’s standing rule applied to the oldest format in it, and it is the rule the next five concepts are built on.