Start with a debt, because the debt is the reason this course exists and it is a single sentence in a file you may have read. The execution course's dependency concept explains that a loop unrolls into “multiples of the SIMD width”, and its latency concept says a different shape does “the same work in SIMD”. Both use the phrase as a known quantity. Across the five CPU-architecture courses before this one, a word-boundary count finds lanezero times, XMM, YMM and ZMMzero times, and emmintrinzero times.
And the other side of the same debt is stranger. AVX-512 appears in five files, all of them the ISA course, and every one of them is about the 0x62 EVEX prefix and the BOUND instruction whose opcode it stole from. That course will teach you to decode the byte that introduces an AVX-512 instruction and will not tell you what the instruction does. A vocabulary that appears only in a prefix table has been taught as a decoding problem, not as a machine.
$ grep -rlwi -e XMM -e YMM -e ZMM -e lane -e emmintrin content/src/*.ch | wc -l 0
So the first thing this course owes you is a definition, and a definition in this collection is a table with integers in it rather than a paragraph. Here it is — and it is computed by the artifact at run time from sizeof, not copied out of a manual, so it is a fact about this build rather than a fact about a document:
$ ./simdbench | sed -n '/register bytes/,/k0-k7/p'
register bytes 64-bit double 32-bit float 32-bit int 8-bit int 16-bit int
XMM 16 2 4 4 16 8
YMM 32 4 8 8 32 16
ZMM 64 8 16 16 64 32 <- not in this CPU
k0-k7 64 bits, one per lane of a ZMM. 64 of them per ZMM, 512 per
architectural ZMM. Not in this CPU.
A vector register is a number of bytes. A lane is a division. That is the whole content of the register, and everything else in this course is a consequence of it. The execution course's phrase has a referent: the SIMD width is 16, 32 or 64 bytes, and “multiples of the SIMD width” means “multiples of that byte count, in elements of whatever type the array holds”.
Read the table by columns and every row is arithmetic. Doubles are 8 bytes, so an XMM holds 16/8 = 2 and a YMM holds 32/8 = 4. Two lanes was not a choice anybody made. It is the only answer that fits, and the same division gives 4 floats, 8 floats, 4 int32s, 16 int8s and 8 int16s in a YMM without a single one of them being a design decision.
Now the trap, and it is the one the whole course turns on. The table has a column that says how many elements an instruction touches, and it does not have a column that says how much faster that makes anything. Those are different columns, and the second one is a measurement, and here it is:
$ ./simdbench | sed -n '/working set/,/beyond L3/p' L1 0 KiB 2 KiB 0.719 0.361 0.185 3.89x L1 -> L2 32 KiB 96 KiB 0.799 0.461 0.357 2.24x beyond L2 256 KiB 768 KiB 0.827 0.507 0.431 1.92x beyond L2, inside L3 2048 KiB 6144 KiB 0.831 0.508 0.429 1.94x beyond L3 8192 KiB 24576 KiB 1.531 1.442 1.340 1.14x
Four lanes. The first row says that buys 3.89× and the last row says the same four lanes buy 1.14×. The register did not change. What changed is whether the three arrays fit in a 32 KiB L1, and the answer to “how fast is four lanes” turns out to be “it depends, and here is the dependency”.
The most useful way to hold this is as a division. The speedup is bounded by how much of the loop the width actually divides:
That is the model. The rest of the course is three elaborations of it: the compiler is one way of getting the issue ceiling (concept 2), the horizontal add is the one place the width does not divide anything (concept 4), and the gather is a case where the width makes the addresses worse (concept 5).
Three things the table cannot say, each of which is a trap, and each of which the artifact has to state as a limit because on this machine it cannot be measured.
The manuals describe the upper-ZMM state: on every x86 CPU since 2013 the upper 256 bits of a ZMM are aliased onto the YMM, and on the parts with 256-bit-and-down registers a VEX-encoded instruction zeroes the upper bits of its destination. So writing a YMM and reading a ZMM's low half are the same register, and the 512-bit encoding does not give you a second independent register for the price of one. This artifact cannot measure any of it — five CPUID leaves, all zero, printed in section 0 — so it is quoted and not claimed, and the limits block says so.
On x86-64 the width is a property of the encoding: VEX encodes 128 or 256, EVEX encodes 512. On AArch64 NEON it is 128 bits and always has been, so a portable NEON loop over doubles is 2-wide and there is no wider option in the base architecture at all. On SVE and on RISC-V V the width is chosen by the implementation, and the same binary runs at a different width on a different part. The three-architectures concept is entirely about this, and it is quoted rather than measured.
This is the one the rest of the course pays for, so it is worth getting exactly right even though it cannot be measured here. A mask register is k0–k7, 64 bits each, one bit per lane of a 512-bit register. A masked load uses a set bit to say which lanes to take; a masked store uses a set bit to say which lanes to keep. It is not a modifier on the instruction — it is a full operand with its own register file, and it is the reason AVX-512 has essentially no tail problem. The last concept of module 3 measures the tail this machine has to handle without one, and measures the scalar answer as free.
And the honest close on the table: none of these five courses had any reason to teach this. Each of them had a single core and a single value at a time, and a single core processing one value at a time does not need the word. It is the same shape of debt as “SMT siblings” in the memory course, and it is discharged the same way: by printing the definition in the artifact’s own output rather than in a changelog.
The harness asserts this table exactly, and it is worth seeing why that is a legitimate exact assertion when every other number in the artifact is a shape. Do the division yourself for the two columns that are not obvious:
Now the part that is not a division, and it is the part a code generator has to know. The width is in the instruction encoding, and the encoding is what makes the two load families different.vmovapddemands a 32-byte aligned address; vmovupd demands nothing. Those are two different opcodes with two different alignment contracts, and the difference between them is not a performance hint — the next section of the course measures the aligned one faulting on an address the unaligned one handles silently.
So here is the whole thing as a compiler backend would want it, in the order it would want it:
Steps 1 to 3 are the whole of vectorisation and they are a page long. Steps 4 and 5 are the rest of the course, and they are where every wrong “it is 4× faster” claim in the world comes from — because step 4 is silently false for a gather and step 5 is silently false for a sum.
cd courses/simd/assets/samples && ./build_samples.sh, then ./simdbench | sed -n '/register bytes/,/k0-k7/p'. (Expect thirty-two bytes for YMM and four lanes of doubles, exactly. Then change the 64-bit double column to 32-bit float in sec_register without changing the code that prints it, rebuild, and watch the harness fail on that one check and no other — which is what an exact structural assertion is for, and is the only kind of check in this artifact that can tell you which cell you broke.)width, vector and wide with word boundaries. (Expect a handful of hits, none of which is a definition, and expect at least one of them to be about a cache line's width rather than a register's. The exercise is not to find the term — it is to notice how comfortable you were with it before this concept.)pixels[i * stride + offset], a sparse matrix multiply, a particle gather. (Expect that hand-unrolling it by four scalar loads and assembling the vector is competitive with a wide instruction, and on this machine it is 2.8× faster. If you find a case where the hardware gather wins, it will be one with a regular stride — and the answer to that is a segment load, which is a different instruction on a different architecture and is the last third of module 3.)Backwards, this concept is the definition the execution course's dependency concept and its latency concept both assumed. Read them after this one and the phrase “multiples of the SIMD width” will have a referent. The ISA course's length concept is where the VEX and EVEX encodings that decide which of these three widths an instruction gets are laid out, and it is worth reading in that order — this concept gives the registers, that one gives the bytes that select them.
Forward, the compiler concept takes the 4-wide arm from the table above and asks the question this concept deliberately did not answer: if the width only buys 3.89× in L1 and 1.14× out of it, which of the two ceilings is binding in your code, and how do you know? It answers it with a disassembly rather than a speedup, and it retracts two claims this course made about the compiler in its own first draft.
Outward, the connection that will matter longest is to a code generator. The memory course's writes concept already showed a loop with three loads and a store per iteration and called it a shape; this course says the shape has a width, and that the width divides the memory operations by four only in the instruction count sense. The WebAssembly course's instruction concept has the simd proposal quoted as a specification and never connected to a register — and that proposal is the one place in the collection where a 128-bit vector type is defined for a machine that does not have one, which is the cleanest possible statement of the abstraction this table is about.