There is exactly one thing a vector arithmetic instruction does that a scalar one cannot, and it is worth saying in one sentence: it writes each lane from the same lane of its inputs and reads nothing from any other lane. Every wide instruction in the x86, NEON and RISC-V V sets obeys this. It is the whole reason a vector loop can be faster, and it is also the reason there is exactly one instruction in the whole family that behaves differently — and that one is the subject of the next concept.
So why does this need a concept? Because “it is elementwise” is not a performance property, and the fastest and slowest vector loops in the world are both perfectly elementwise. Here is the whole of the difference, in one table, with nothing changed but the number of accumulators:
$ ./simdbench | sed -n '/same arithmetic, different independence/,/^ and the two/p' same arithmetic, different independence ns/iter speedup checksum ONE accumulator, 4-wide FMA, 16 iterations 28.51 1.00x 15.000000 FOUR accumulators, 4-wide FMA, 4x4 iterations 7.13 4.00x 12.000000
Same instruction. Same width. Same number of them. One is 4× the other. The only difference is whether the second FMA needs the answer to the first. Nothing about the instruction changed; something about the data changed, and the data is what the compiler cannot see when it decides whether to vectorise.
That is the sentence the previous concept was reaching for and could not state, and it is the one worth taking into a compiler backend. Independence is a property of the data, not of the instruction. Sixteen independent chains hide a 4-cycle multiply latency completely. One chain of sixteen is sixteen dependent 4-cycle latencies, and the wide register does nothing about it.
Four bodies over the same 64 doubles, differing in one arithmetic instruction. The first is a copy — two loads and a store, no arithmetic at all — and it is an arm rather than a preamble because “what does the arithmetic cost” has no answer without it.
$ ./simdbench | sed -n '/body /,/FMA, store/p' body ns/iter vs copy checksum load, store (ZERO arithmetic instructions) 7.13 1.00x 32.000000 load, ADD, store (one add) 8.02 1.12x 640016.000000 load, MUL, store (one multiply) 7.63 1.07x 0.000000 load, FMA, store (one multiply-add, one instruction) 11.14 1.56x 32.000000
Read the three arithmetic rows against each other first. An add and a multiply are within 7% of each other, and that is the shape of a loop that is not waiting for the arithmetic. Their bodies differ by one instruction and a multiply is no dearer than an add, because both of them hide inside what the two loads and the one store already cost.
That is the mechanism behind the FMA row in the previous concept's table, seen from this side. FMA halves the arithmetic instruction count from two to one, and the time does not move, because the arithmetic was never the thing being waited for.
This is the trap in the table and it has a name, because it is the same confound that concept 2's arm F exists to remove. The FMA body reads three arrays and the other two read two. The row with fewer instructions has more memory operations and costs more. Reading it as “FMA costs 1.56× a multiply” is wrong, and it is a very easy mistake to make because the FMA is genuinely the better instruction.
And the checksum column is doing more work than it looks like it is. The four bodies leave four different numbers: a copy leaves 32, the add leaves 640016 (it is a divergent recurrence), the multiply leaves 0, and the FMA leaves 32 because it is the same fixed point the rest of the artifact uses. That is why the checksums are printed and not asserted equal — and why an arm that silently computes the wrong thing shows up here as a wrong number rather than as a plausible timing.
Now the two arms from the top of this concept, and the mechanism they differ by. Both run sixteen 4-wide FMAs per iteration over the same data. One accumulates into a single register; the other into four:
Measured: 4.00×. And note the second thing in that table — the two checksums are different (15.0 and 12.0), which is how you can tell at a glance that these are not secretly the same loop. A harness that asserted they must match would have been asserting a bug.
Three consequences, and the third is the one that reaches into the next concept.
And the one instruction that is not elementwise, stated precisely because it is the exception that proves the rule. A horizontal add takes one vector register and produces a scalar by adding across its lanes — that is, lane 0 plus lane 1 plus lane 2 plus lane 3. It is the only common instruction that reads across lanes, and it is the only place in this course where a 4-wide loop is slower than a scalar one. The next concept measures it, and the number is worse than you would guess.
Everything above is one loop, so here is the general case — the order matters, because each step can fail and a failure at step 3 kills steps 1 and 2.
Step 4 is the one people skip and it is the one that decides whether a change is worth reviewing. A 1.14× speedup is not nothing — it is worth having — but it is worth having only if the loop is going to be read and maintained anyway, and the review cost of intrinsics in a codebase that has none is not zero.
And a worked example of step 1 failing, because it is the interesting failure. Consider a prefix sum, for (i) s[i] = s[i-1] + a[i]. Every element depends on its predecessor, so it is not elementwise and no width helps. The standard fix is not a wide instruction — it is to restructure into a scan, which computes partial sums in a tree so that the work at each level is independent, then combines the levels. That is a real algorithm with a real correctness argument and it is a different program, not a different instruction. When a loop cannot be vectorised, the fix is almost never a wider register; it is a different loop.
That is worth contrasting with the reduction, which is also not elementwise and which does have a wide-instruction answer — because a reduction can be done in parallel across independent partial accumulators and folded at the end. The next concept measures what that fold costs, and the answer is “a horizontal add of four doubles costs about as much as the vector operation that produced them”, which is why the fold happens once and not per iteration.
simdbench.c and count the arrays each one reads. Then try the honest comparison: time a mul-then-add body against the FMA body with the loads matched, and expect FMA to be worth very little.)dep_one to use two accumulators, then four, then eight, and rerun. (Expect 1.00×, about 2×, 4×, then a plateau — the machine has a fixed number of multiply pipes and once you have more chains than pipes the extra ones queue. Finding the plateau is the exercise: it is the number of arithmetic units, and on this machine it is small enough that four is enough.)sum += a[i] * b[i] where a compiler has vectorised the multiply but not the sum. (Expect the compiler to have handled most of them, and expect the ones it handled badly to be the ones where the accumulator is a double rather than a float, or where the loop is a reduction over a struct field. A one-chain vectorised loop is a loop where the wide register bought the multiply and charged you for the chain.)Backwards, the compiler concept asked what a compiler does with a loop it cannot prove anything about, and this concept is the answer to why the question matters: the vectoriser's job is to find independent work, and aliasing is what stops it looking. The width concept supplied the register table whose whole content is a division, and this concept is the first place that division stops being enough.
Forward, the reduction concept is the exception that this concept sets up. Everything here is elementwise, and a reduction is the one operation that is not — it is the only place the lanes must talk to each other, and the artifact measures what that costs and finds it costs about as much as the vector operation itself. The boundaries concept then takes the load and the store, which the four-body table above showed is the real cost, and asks what happens when the addresses stop being nice.
Outward, this is the concept where the execution course's central lesson is finally stated in its own terms. The dependency concept teaches that a chain of dependent operations is bounded by latency and a set of independent ones by throughput; this concept measures that on a 4-wide register and gets 4.00×, which is exactly what “independent” means when the width is four. The latency concept is where the multiply's latency is measured — and this course assumes it rather than re-measuring it, because re-measuring it would be a fourth course's job and not a fourth course's lesson. The memory course's writes concept is where three loads and a store per iteration was already a shape; this concept is the table that says what that shape costs and that the arithmetic in it is nearly free.
And the practical consequence, which is the one to carry out of here: when a loop does not vectorise, the question to ask is not “how do I make the compiler vectorise it” but “is there independent work in here at all”. A prefix sum has none, and the answer is a different algorithm. A dot product has some — four partial sums, folded once — and the answer is intrinsics. Telling those two apart is the skill, and it is a question about the data, not about the compiler.