x86-64 has sixteen general-purpose registers. Every legacy instruction has three bits per register field, which names eight. Something has to bridge that gap, and the answer is a single byte that was already spoken for.
$ ./build_samples.sh 2>&1 | sed -n '/I2/,/I3/p'
the four bits, W R X B, in 0x40..0x4F:
89 c0 89 c0 mov eax,eax
^ no REX 32-bit operands
44 89 c0 44 89 c0 mov eax,r8d
^ REX 0x44 R=1 32-bit, reg becomes r8d
4c 89 c0 4c 89 c0 mov rax,r8
^ REX 0x4c W=1 R=1 64-bit, reg becomes r8
49 8b 00 49 8b 00 mov rax,QWORD PTR [r8]
^ REX 0x49 R=1 B=1 r/m becomes r8
45 8b 04 24 45 8b 04 24 mov r8d,DWORD PTR [r12]
^ REX 0x45 X=1 B=1 base AND index become r12
Read those five lines as a single mechanism. REX is four bits — W, R, X, B — and three of them do exactly one job: extend a three-bit field to four bits. One extends the operand size. Three extend the three register fields — R for the ModRM reg field, X and B for the address.
And the rule that catches everyone, including this course’s author before it was measured: the extension bit ADDS 8. It does not select “the ninth register”. So R=1 with reg=000 is r8, and with reg=100 it is r12. The field stays a 3-bit number and the bit supplies bit 3.
That is a strange design and it is worth asking why, because the answer explains the second rule. The alternative was to widen the register fields to four bits, which would have meant an entirely new encoding — and the whole point of 64-bit mode was compatibility. An old decoder, handed a 64-bit binary, should ideally still decode the parts it understands. So Intel kept the legacy encoding intact and added a byte that widens it, rather than widening the fields in place.
Which produces the second rule, and it falls straight out of that:
And that is not a stylistic rule. It is a state-machine rule, and it exists because a REX byte is only meaningful when it is adjacent to the thing it modifies. If 66 could follow 48 and still be a prefix, the decoder would have to remember an unbounded number of prefix bytes and their order — but the hardware has no room to remember that, so the rule is enforced in the encoding instead.
There is a third property that falls out of the “adjacent” design and is easy to miss:
So the three REX rules are: it adds 8, it must be last, and only the last one counts. All three are consequences of one decision — prepend a widening byte rather than rewrite the fields — and all three are things you can get wrong in a compiler, a bootloader, or a hand-written assembler.
Every bit measured, one at a time, against the oracle:
$ python3 crosscheck.py 2>&1 | sed -n '/^I2/,/^I3/p' ok all 256 ModRM bytes decode as exactly one instruction (that is I3) ok REX extension in 44 89 e0 -> r12d ok REX extension in 4c 89 e0 -> r12 ok REX extension in 49 89 e0 -> r8 ok REX extension in 49 8b 00 -> [r8] ok REX.R=1 with reg=000 is r8, NOT r9 ok a bare REX prefix does not change the instruction ok 66 then REX is ONE instruction ok REX then 66 is TWO objdump entries ok x86dec agrees on the 66-then-REX case ok x86dec REFUSES REX-then-0x66, which is the architectural reading
Now the part that is worth more than any of those checks, because it is a limitation of the instrument rather than of the ISA. The fifth row of the I2 output in the build script is this:
A LIMITATION OF THE ORACLE, measured and worth knowing: the last row has REX.X=1, so the SIB index (100 with X=1) should be r12 as well as the base. objdump prints only [r12] -- it does not apply REX.X to the SIB index field. x86dec.py DOES apply it, so the two disagree here, and the crosscheck compares instruction BOUNDARIES rather than operand text for exactly this reason.
45 8B 04 24 has REX.X=1 and REX.B=1. The base is 100, so with B it is r12. The index is also 100, so with X it should be r12 too — the instruction is mov r8d, [r12+r12*1]. The oracle prints [r12]: it applied B and ignored X.
This is a real bug in the tool, and it has a consequence for how the course is verified. If the crosscheck compared operand text, it would fail here — with the decoder right and the oracle wrong. So the crosscheck compares instruction boundaries and lengths, which both readers get right, and the operand text is printed for a human to read rather than asserted.
That is the general lesson and it is the same one the hardening course learned about the wrong output stream: a check that fails for the wrong reason teaches you to ignore it, which is worse than having no check. So the right move is to narrow the check until it tests something both tools are reliable about, and to write down the part that is not covered rather than quietly dropping it.
Each rule has a characteristic bug, and each bug is silent. That is what makes them worth memorising as a set rather than as a definition.
The third is the dangerous one and it has a real historical shape. Tools that rewrite machine code — emulators, binary patchers, JIT compilers that emit x86 directly — naturally prepend a prefix to widen something. If the instruction already carried a REX, the prepended one is discarded and the widening silently does not happen. The emitted bytes are valid, the code assembles, and the value is wrong.
$ python3 x86dec.py --why 4c 89 e0 | head -6
offset 0, 3 bytes: 4c 89 e0
mov r12,eax
. byte 4c at 0 is a REX prefix: W=1 R=1 X=0 B=0
. the three extension bits each ADD 8 to a 3-bit field,
. so R=1 with reg=000 is r8 -- not r9
. byte 89 at 1 is a one-byte opcode
. byte e0 at 2 is the ModRM byte: mod=3 reg=4 rm=0
. mod=11: the r/m field names a REGISTER, so no address
. bytes follow
That derivation is the whole concept in six lines. 4c is REX with W=1 and R=1. 89 is MOV r/m, r. The ModRM is e0 = 11 100 000: a register form, reg=4, rm=0. With R=1 that is reg = 4 + 8 = 12 — r12, not r8, because the bit is added to the three-bit field rather than selecting a group. And W=1 makes the operands 64-bit. The instruction is mov r12, rax, four bytes, and every one of those conclusions came from two bits of one byte.
One more thing the derivation shows, and it is the bridge to the next concept. W=1 changed the operand width without changing a single bit of the ModRM.4c 89 e0 and 44 89 e0 have identical opcodes and identical ModRM bytes, and they are different instructions operating on different widths. The next concept is about that byte, and this is the first evidence that the opcode and the ModRM are not independent knobs.
$ cd courses/isa/assets/samples $ ./build_samples.sh 2>&1 | sed -n '/I2/,/I3/p' $ python3 x86dec.py --why 4c 89 e0 $ python3 crosscheck.py 2>&1 | sed -n '/^I2/,/^I3/p'
Then break each rule deliberately, which is the better exercise because the breakage is instructive:
1. Prove the ADD-8 rule by brute force. For each
of the 8 reg values and both values of R,
emit the instruction and read the register
name:
for r in $(seq 0 7); do for R in 0 1; do
rex=$((0x44 + 2*R))
printf '%02x 8b c0\n' $rex # rm=0
done; done
Then print the sixteen register names in
order and check they come out rax rcx rdx rbx
rsp rbp rsi rdi r8 r9 r10 ... r15 -- and
NOT rax ... rdi r9 r10 ...
2. Now the order rule, as a table. Every pair of
prefix bytes, both orders, decoded:
for a in 66 67 f0 f2 f3 2e 36 3e 26 64 65 48 4c; do
for b in 66 48 4c; do
printf '%s %s 8b c0\n' $a $b
done
done | xxd -r -p | ...
(Some orders are one instruction, some are two,
and the rule predicts which: a legacy prefix
followed by anything is one; anything followed
by REX is one; REX followed by a legacy prefix
is two. Find a case that violates the rule, if
one exists -- the course did not find one and
does not claim there is none.)
3. The double-REX trap, concretely. Take a real
instruction that already has a REX, prepend a
different REX, and show that the first is
discarded:
4c 89 e0 mov r12,rax W=1 R=1
48 4c 89 e0 mov r12,rax the 48 is a no-op
4c 4c 89 e0 mov r12,rax only the last counts
But now try to actually CHANGE the width by
prepending, and watch it fail:
89 e0 mov eax,eax
48 89 e0 mov rax,rax works: REX first
44 48 89 e0 mov rax,rax the 44 is DROPPED,
so the operand is
64-bit when the
emitter wanted 32
(That last one is the bug in miniature: the
emitted bytes are valid, the instruction
assembles, and the width is wrong. It is the
failure mode a binary rewriter has to test
for, and it produces no diagnostic.)
4. Check the oracle's REX.X gap against a second
independent source so you are not trusting
one tool. Build the same instruction and
count the registers the SIB can reach:
45 8b 04 24 REX.X=1 B=1
41 8b 04 24 X=0 B=1
What SHOULD the first be? ([r12+r12*1]) What
do the tools print? (objdump: [r12] both
times; x86dec.py: [r12+r12*1] then [r12].)
The two tools differ, one of them is wrong,
and the way to tell which is to reason from
the field layout rather than to ask a third
tool that might share the same bug.
5. Finally: audit the whole REX space. For all
sixteen REX values against a fixed ModRM, what
changes and what does not? Print the sixteen
results and check that W is the only bit that
changes the operand WIDTH, and that X and B
change nothing at all when the ModRM has
mod=11 and no SIB.
for rex in $(seq 64 79); do
printf '%02x 89 c0\n' $rex
done
(Expect: W flips between 32-bit and 64-bit
forms, R changes the destination among the
high registers, and X and B are INVISIBLE --
because there is no SIB and the r/m is a
register field that B does extend. Check
whether B is visible. It should be: B extends
ModRM.rm, and rm=0 here. If your output says
it is not, you have found the same class of
bug as the oracle's.)
Exercise 3 is the one that makes the concept a habit, and the failure it produces is the most instructive thing in this unit. “The emitted bytes are valid, the instruction assembles, and the width is wrong” — there is no diagnostic, no crash, and no way to notice without reading the bytes back. That is the signature of a whole class of binary-manipulation bug, and it is why the hardening course’s posture reader exists: every serious tool that rewrites machine code has to be checked by reading the file, because the tool cannot check itself.
Exercise 4 is the one that teaches the method rather than the fact. When two tools disagree, do not reach for a third — work out the answer from the field layout and then say which tool is wrong. Here the field layout settles it: REX.X is documented as extending the SIB index, so the oracle is wrong and the decoder is right. Reaching for a third tool would have produced a majority vote instead of a reason, and majority votes are how a shared bug becomes a fact.
The connection to the PIC concept in the object course is the most direct in the chain, and it is a debt. That concept measured that position independence needs a base register because an absolute 32-bit address cannot be relocated, and concluded that the fix is RIP-relative addressing. It could not say how RIP-relative addressing is encoded, and this is the answer: it is REX.B=1 selecting rIP as the base, and a disp32 after the ModRM. The B bit this concept measured is the bit that concept’s mechanism needs. Three courses, one bit.
The connection to the PIC-violation concept is about the cost. That concept measured that R_X86_64_32S overflows above 0x80000000 — which is a consequence of W selecting a signed 32-bit displacement rather than an unsigned one, so that a RIP-relative access can reach backwards. This concept is where that choice lives, and it is the reason the field is disp32 and not disp64: the encoding-limits concept measured the price of position independence and this is the mechanism paying it.
Two connections outward. The CET concept measured that a binary can contain endbr64 and request no enforcement, and the previous concept measured the encoding: F3 0F 1E FA, a reinterpretation by prefix. REX is the other kind of prefix — it widens rather than reinterprets — and the distinction is the whole taxonomy of prefix bytes: operand size, address size, REX widening, lock/rep semantics, segment override, and reinterpretation. And obj-arch-table again, because the question it asked is still the live one: it wanted to know how many bytes an instruction is, and the answer turns out to be “it depends on a bit in a prefix, so you have to decode the prefix first.”
One limit, and it is a real one rather than a formality. Nothing here was measured on a second architecture, and the REX design is specifically a solution to the 32-to-64-bit transition. AArch64 has no REX because it has no legacy mode to stay compatible with; its register-to-field mapping is fixed at five bits in the base encoding. So the pattern here — a prefix that widens a field rather than a field that is wide — is portable, and the specific mechanism is not. The DWARF frame course covers the AArch64 side of register encoding, and this course deliberately stops at x86-64 rather than asserting a symmetry it did not measure.