Almost every conversation about SIMD is the same conversation, and it is always conducted in terms of intrinsics: intrinsics beat the compiler, or the compiler beats intrinsics, or the compiler would have done it if you had written it differently. That conversation cannot be settled with a stopwatch, and this concept is the one where this course says so and then does the thing that settles it.
Start with the result, because it is the first thing a reader expects and it is a null:
$ ./simdbench | sed -n '/THE COMPARISON, as SPEEDUPS/,/AND THE FINDING/p'
THE COMPARISON, as SPEEDUPS over the floor B (higher is faster):
2 lanes (C) / 1 lane (B) 2.00x
4 lanes (D) / 1 lane (B) 3.90x <- the lane count is 4
4 lanes with FMA (E) / 4 lanes without (D) 1.00x <- FMA's contribution
the hand-written 4-wide (E) / the compiler (A) 1.00x
one fewer memory op (F) / three loads (E) 1.12x <- SLOWER, and that
is the finding: a memory operand buys a shorter instruction and
costs a longer dependency. Instruction COUNT is not the cost.
The auto-vectorised arm of a plain C loop and a hand-written 4-wide intrinsic loop over the same 64 doubles came out identical to two significant figures. A compiler will vectorise a straightforward loop at -O2 these days, and it will do it about as well as you would. The interesting part is the last row, which is slower: the arm that removes a whole load by folding it into the FMA as a memory operand.
And then the part that made this course retract two of its own claims. The same compiler, handed the same statement through a different declaration, produces a loop that is 35× slower than the vectorised one and slower than the scalar floor:
L1 0 KiB 2 KiB 0.719 0.185 3.89x 3.90x 0.11x
beyond L3 8192 KiB 24576 KiB 1.531 1.340 1.14x 1.18x 0.23x
(last two columns: auto-vec through NAMED arrays, and through double *)
The difference between 3.90× and 0.11× is three double * declarations. Not the trip count, not the optimisation level, not a missing -march. That is a fact about code generation, and it is the reason this concept exists: you cannot find it with a stopwatch, because both arms are timings and the timings only tell you that one was fast.
To find out what the compiler actually does, you look at what the compiler actually emitted. The artifact compiles four forms of one statement with the same flags, crossing two independent variables:
And build_samples.sh appends the disassembly of all four to the recorded output, because a claim about what a compiler did is a claim about bytes and the only way to check one is to read them:
$ tail -30 simdbench.out
named_const:
1 vfmadd132pd
=> VECTORISED: 1 packed-double, 0 scalar
named_var:
1 vfmadd132pd
1 vfmadd132sd
=> VECTORISED: 1 packed-double, 1 scalar
ptr_const:
=> NOT VECTORISED: 0 packed-double, 1 scalar
ptr_var:
=> NOT VECTORISED: 0 packed-double, 1 scalar
Read it as a 2×2 and the result is unambiguous and it is not the one this course was written to say:
That is aliasing, and it is worth being precise about the word because “aliasing” is usually a claim that two pointers are equal. Here it means the compiler cannot prove they are not. One restrict qualifier, or three arrays instead of three pointers, is the entire difference between the two columns of that table.
The first version of this course reached the three arrays through pointers, measured the auto-vectorised arm sitting on the scalar floor, and wrote this:
R7. "The compiler declined to vectorise because the trip count is a
runtime variable." WRONG, AND IT WAS AN INFERENCE FROM A TIMING.
It was an inference from a timing, and the disassembly says something else entirely. The named form vectorises at both trip counts. What the trip count costs is an epilogue, and the epilogue is free; what the pointers cost is the entire vector loop, and that is catastrophic. Two different mechanisms, one of them harmless and one of them ruinous, told as a single tidy sentence because a tidy sentence is what a number going the right way produces.
And the same sentence carried a second error, which is the one this course was really about. The first draft also claimed that the compiler beats intrinsics “because it hoists the loop-invariant loads of B and C”. The disassembly shows B and C loaded inside the inner loop on every one of the sixteen iterations. They are not hoisted. What the compiler does is put A into the FMA as a memory operand — which is arm F in the table above, and the artifact measured arm F and it is slower. So the story was wrong twice: the mechanism is not hoisting, and the mechanism it gets confused with is not free.
Now the part that makes it a course rather than a bug report. The control arm lied first. The point of the pointer column is that it is a control: the same statement, the only difference being a declaration. So it was written as
/* the first control, which was not a control */
static double *gA = bA, *gB = bB, *gC = bC;
— and that is the same three arrays wearing a pointer costume. gcc folded the static initialisers away, proved the three streams distinct again, vectorised the control, and the control came out at 3.91×, indistinguishable from the arm it was supposed to contradict. The measurement was not wrong. The control was not a control.
The fix was to assign the pointers in main() and reach them through noinline accessors, so the compiler cannot see through them. The arm then emits one scalar FMA and no vector one, and lands where the disassembly says it should. And the harness now has a check for the retraction itself, so a future edit that restores the static initialisers turns one group red by name.
One thing the control is not, and it is worth saying because the honest reading matters: it is not a clean 35× of pure vectorisation. The noinline accessors put three calls in the inner loop, so the gap between the two columns is larger than the aliasing effect alone. The defensible claim is “catastrophically worse”, not a factor.
Four things, in the order they are worth doing.
And the reason this is worth a whole concept rather than a footnote: the same reasoning applies to every “the compiler did not do the obvious thing” report you will ever be handed, in every language. The compiler's behaviour is a fact about a specific program, a specific version, and a specific set of flags, and a timing is not a way of establishing it. The disassembly is one command and it settles the question; the stopwatch starts a four-hour argument.
There is one more reading of the table that is worth making, because it is the one that survives being wrong. The auto-vectorised arm and the hand-written arm are within a percent — but that is on this loop, on this machine, at this working-set size, and the previous concept measured the same two arms at five working-set sizes where the whole question stops mattering. A compiler that matches your intrinsics on an L1-resident loop is not telling you anything about an L3-resident one. Both facts are in the same artifact and they are about different ceilings.
simdbench.c, change static double *gA, *gB, *gC; to static double *gA = bA, *gB = bB, *gC = bC; and delete the three assignments in sec_ceiling. Rebuild, run, rerun the harness. (Expect the pointer column to come back to about 3.9× and group D to fail on the control checks by name. Write down which checks failed before you predict which would — the gap is usually a check you did not know existed. Then put the assignments back.)./verify_codegen.sh /tmp/whatever.out and then objdump -d --no-show-raw-insn the object it made. (Expect the 2×2 above, and expect it to be exactly the same on your machine. If it is not, your compiler has a different cost model and the first thing to check is whether -O2 still means -ftree-vectorize in your version. Note which of the four forms you would have got wrong by timing alone.)restrict to a real function of yours and disassemble it before and after.(Expect the scalar version to become a 4-wide vfmadd loop with a scalar epilogue, and expect the change to be one keyword. Then check the call sites: restrict is a promise that the three pointers do not overlap, and the compiler will not verify it. An overlapping call is undefined behaviour you wrote yourself.)-fopt-info-vec-missed on a loop you expected to be vectorised.(Expect a per-line reason, and expect the reason to be a cost-model judgement or an aliasing one rather than “could not vectorise”. This is the direct replacement for the stopwatch argument: it takes ten seconds and it is right.)#pragma GCC unroll, or force the alignment. (Expect a small number of cases and expect them to be about the epilogue or the alignment, never about the arithmetic. If you find one where the compiler emits a worse algorithm than you wrote — a different reduction order, a lost fusion — that is the interesting one, and it is worth writing down with the -fopt-info output beside it.)Backwards, the width concept is the one whose table made this question necessary. It deliberately refused to say how much a wide instruction is worth — and this concept answers it in two halves: on an L1-resident loop the answer is “the full width” and the compiler gets it for free, and on anything else the answer is “almost nothing” and the compiler getting it for free is the point.
Forward, the shapes concept answers the question this one raises and does not: if the compiler's arithmetic and yours are identical and the loop is not waiting for the arithmetic anyway, what is a vector loop waiting for? The answer it measures is the load and the store, and the independence of the lanes, and those two facts are the last word on why the collapse in the previous concept happens where it does.
Outward, the connection to a code generator is direct and it is the point of the course so far. The ISA course's length concept is where the encodings are, and this concept is the place where you find out which one the compiler picked and why. The privilege course's harness is the model for what the group D checks do, and it has the same three rules plus four documented failures of its own; reading it next to this one is how you learn that a check written to defend a claim rather than a measurement is a check that launders the claim — which is precisely the error R7 and R8 are. And the multiprocessor harness is the immediately preceding instance, whose first check asserted a difference its own data contradicted.
One outward-facing consequence, and it is the most useful thing in the concept. On a machine where the compiler declines to vectorise, the difference between a fast loop and a slow one is frequently a type declaration. That is a sentence about code generation that holds in every language in this collection's orbit, and it is worth carrying into the WebAssembly course's instruction concept, where the same aliasing question decides whether a simd type is worth using at all.