Every performance argument in this collection has been static so far. A linker adds a relocation, a hardening flag costs a byte, a PIC access needs two fixups — all countable from the file, none of them a time. This is the concept where the counting becomes a measurement, and where two numbers that are always confused turn out to differ by a factor of three.
$ ./cycbench 2>&1 | sed -n '/3. LATENCY/,/4. THE FLAGS/p'
3. LATENCY AND THROUGHPUT
-------------------------
body min ticks vs floor
loop floor (empty body) 0.6678 1.00x
1 dependent add 0.8774 1.31x
2 dependent adds 1.9690 2.95x
4 dependent adds 3.6190 5.42x
8 dependent adds 7.4115 11.10x
1 independent add 0.6952 1.04x
2 independent adds 0.6784 1.02x
4 independent adds 0.8367 1.25x
6 independent adds 1.3272 1.99x
MARGINAL cost of one more operation, floor removed:
dependent chain 2 adds +1.0916 4 adds +0.8250 8 adds +0.9481
independent 2 adds -0.0168 4 adds +0.0791 6 adds +0.2453
a dependent chain costs 3.87x an independent operation.
Two rows carry the concept. Eight dependent adds cost 11.1× the empty loop; six independent adds cost 2.0×. Same instruction, same encoding, same machine, same clock — and the only difference is whether each one needs the previous one’s answer.
And the marginal row is the real finding, because the totals are contaminated by the loop. Add one more link to a dependent chain and it costs about 0.95 ticks. Add one more independent operation and it costs about 0.25. That ratio of roughly 2.7–3.9× is the execution width, measured, without a single cycle count anywhere in the argument.
Two questions that are always asked together and are never the same question:
So why do the two differ, when the work is identical? The model answer is the core can start several operations per cycle because their inputs are ready at the same time:
Which gives the practical rule, and it is the most useful thing in this concept:
And the honest caveat that the measurement makes unavoidable. The totals in the table above are not the latency and the throughput; they are the totals, which include the loop. One operation and two independent operations both measure at the floor — 0.695 and 0.678, with the difference inside the noise — because with a loop overhead of about 0.67 ticks, one or two operations hide entirely underneath it. Only from four upwards does the body start to dominate, and only the marginal costs are the latency and the throughput.
That is not a flaw in the measurement; it is the measurement, and it is the reason the marginal row exists. A benchmark that reported the totals as “latency” and “throughput” would be reporting the loop.
Before the numbers above, there was this:
$ ./lat # the first version, one add per iteration loop floor (nop body) 1.109 ticks/iter 1 dependent add 1.084 ticks/iter 2 dependent adds 2.402 ticks/iter 1 independent add 1.012 ticks/iter 2 independent adds 1.174 ticks/iter 4 independent adds 1.314 ticks/iter 6 independent adds 2.363 ticks/iter
One dependent add measured faster than an empty loop. That is impossible if you are measuring the add, and it is a very good sign that you are not.
What was being measured is the loop. The body is add plus dec plus jnz, and the dec→jnz pair is a loop-carried dependency: the counter must be decremented before the branch can read its flags. That chain is one link long no matter what the body does, so with a one-instruction body the branch is the bottleneck and the add hides underneath it.
So the first version was not wrong about the machine; it was measuring the loop instead of the instruction, and the giveaway was a result that contradicted arithmetic. A benchmark that reports a negative cost has told you something, and the something is about the benchmark.
And there is a second, larger reason the same table kept disagreeing with itself between runs, which the next concept is entirely about. The two rows for “1 independent add” and “2 independent adds” were not just close to the floor — in some builds the one-operation version measured twice the two-operation version, which is not a rounding error. Byte-identical loops, at different addresses, differing by 2.7×.
The ratio is the number a performance-minded reader wants, so it is worth being precise about what it does and does not give you.
That is a deliberate refusal and it is the honest one. It is easy to pick a plausible-looking core frequency, multiply, and print “the core retires 4 integer adds per cycle” — and on this machine that number would be in the right neighbourhood for the right reasons, which is the most dangerous kind of wrong.
There is a way to close the gap without a PMU, and it is worth naming even though this course did not do it: measure a chain of known length against a clock-domain crossing. A serialising instruction such as CPUID has a documented, large, fixed cost in core cycles; timing N of them against N of an add gives the clock ratio directly. That is a real technique and it would let this course quote a width. It is not here because it was not measured, and a course that measures six things and writes down the seventh it did not measure is more useful than one that quietly rounds a guess into a claim — that is the whole retraction policy of this collection in one sentence.
What is measured, and is arguably the more useful number, is the shape of the cost as a function of chain length:
body min ticks vs floor loop floor (empty body) 0.6678 1.00x 1 dependent add 0.8774 1.31x 2 dependent adds 1.9690 2.95x 4 dependent adds 3.6190 5.42x 8 dependent adds 7.4115 11.10x 1 independent add 0.6952 1.04x 2 independent adds 0.6784 1.02x 4 independent adds 0.8367 1.25x 6 independent adds 1.3272 1.99x
Read the two columns against each other and the shape is unmistakable. Dependent is linear in chain length with a steep slope; independent is nearly flat and then bends. The bend in the independent column is the point where the core runs out of places to start work and the operations begin to queue — which is the same fact as the ratio, seen as a curve rather than a number, and it is the version of the measurement that transfers to a different machine.
$ cd courses/exe/assets/samples $ ./cycbench 2>&1 | sed -n '/3. LATENCY/,/4. THE/p' $ python3 crosscheck.py 2>&1 | sed -n '/^D /,/^E /p' $ python3 crosscheck.py 2>&1 | sed -n '/^E /,/^F /p'
Then do the experiment that turns the ratio into a design tool:
1. Find the knee. Extend the independent
series until it stops being flat -- 6, 8,
12, 16, 24 operations per iteration -- and
find where the slope changes.
# then, for each, print marginal = (t[k]-t[k/2])/(k/2)
(The knee is where the core runs out of
issue slots. It is the closest thing to a
core-width measurement this instrument can make
without a clock, and its POSITION is meaningful
even though its unit is not: it says how much
independent work this core can absorb before
more of it stops helping.)
2. Now the real experiment: same instruction
count, different ILP. Write four versions of
one computation and time them.
(a) a += b*c + d*e; ILP ~2
(b) x=a*b; y=c*d; a += x+y; ILP ~4
(c) four accumulators, unrolled ILP ~8
(d) the same work in SIMD ILP huge
Time all four. Then count the INSTRUCTIONS
in each with the ISA course's decoder:
python3 .../x86dec.py --section .text prog
(Expect (b) and (c) to be faster than (a)
with the SAME or MORE instructions. That is
the whole point: the win is not fewer
instructions, it is more of them running at
once. And you can count the instructions
yourself, which is the nice part -- the
decoder and the timer are two independent
instruments measuring two different
properties of the same binary.)
3. Now break the loop's own dependency, and see
the first measurement's mistake fixed properly:
# instead of dec/jnz, use a pointer that
# walks memory, or a REP, or unroll the
# counter by 8 and subtract 8
for (i = n; i > 0; i -= 8) { body x8 }
(Expect the marginal cost per operation to
DROP, because you have divided the loop cost
by 8. This is the same fix as unrolling, done
to the loop rather than the body, and it is
the reason hand-optimised inner loops are
unrolled in multiples of the SIMD width.)
4. Find a REAL dependency that is not a data
dependency. Take a loop whose iterations are
independent and see what the core still
cannot overlap:
for (i = 0; i < n; i++) out[i] = in[i]*2;
(Expect it to be limited by the LOADS, not
the multiply -- a load has a much longer
latency than an integer add, and the stores
need somewhere to go. This is the bridge to
the memory-hierarchy course, and finding it
here is the point: the arithmetic is not the
bottleneck in almost any real loop, and a
course that only measured arithmetic would
never notice.)
5. Finally: predict, then measure. Before
running anything, write down what you expect
the dependent and independent series to look
like, and in what direction each will bend.
for (i = 1; i <= 8; i *= 2)
for (j = 0; j < 4; j++)
independent[j] = dep_chain_of_i(j);
(Then run it. If your prediction matches, you
understand the mechanism. If it does not, the
disagreement is the interesting part -- and
writing the prediction down FIRST is what
turns a measurement into an experiment. Every
measurement in this course that turned out to
be wrong was wrong because nobody had said
what they expected.)
Exercise 1 is the one that gets closest to a core-width claim without a clock, and the insight is that the position of the knee is meaningful even when its unit is not. “This core absorbs about six independent integer operations before more stops helping” is a real, transferable statement about the machine; “the core is 5.7 wide” is a number that changes with the clock.
Exercise 4 is the one that matters most for a reader who will write real code, and it is the bridge to the next course. A loop that is pure arithmetic is almost never limited by the arithmetic — it is limited by its loads and its stores, which have far longer latencies. This course measured a chain of integer adds and found a clean 2.7× ratio, and that result is real and it is also almost irrelevant to most programs. The memory hierarchy is where the real constraint lives, and the reason to understand latency and throughput here is to be able to recognise when neither is the thing limiting you.
The connection to the ISA length concept is the sharpest in the course, and it is the same relationship read from two ends. That concept measured the mean instruction at 3.99 bytes and the code at 1885 bytes for 472 instructions, and established that instruction boundaries form a chain. This concept measures that a chain of dependent executions is the other kind of chain, and it is the expensive kind: boundaries chained in address order are free, data chained through registers cost a latency each. Two different meanings of “chain”, one verified by arithmetic and one by a clock, and the confusion between them is why the first version of this measurement timed its own loop.
The connection to the encoding-limits concept is a real cost comparison, and it goes the direction nobody expects. That course measured that position independence costs an extra relocation per datum, because RIP-relative addressing needs a disp32 the linker must patch. And the code PIC produces has more instructions but more ILP — a mov of the base followed by an indexed access has more operations available to overlap than the single absolute access it replaces. So the statically-expensive choice can be the dynamically-cheaper one. This course cannot settle that, because it did not measure PIC code, and the honest summary is that the two courses disagree about which cost matters and neither has run the experiment that would settle it.
Two connections outward, both about the shape rather than the number. The next concept asks what a dependency is, and this concept is the evidence that the answer has to include addresses and not just registers. And The ELF header-fields concept said every field exists because the bytes are not self-describing; this concept is the same argument for performance — two byte-identical loops in one binary differ by 2.7× and no field in the file records the difference, because the difference is in the address, which the format does store and the CPU interprets. A linker that knows the address can change the timing; nothing in the file says the timing changed.
One limit, and it is the limit the whole course is built on. No absolute cycle count appears anywhere in these six concepts. The TSC is a time base, the core clock moved 2389–3320 MHz during a single run, and there is no PMU to divide one by the other. Every figure is a ratio or a minimum, and the crosscheck asserts shapes — monotonicity, ordering, bounds — rather than values, because shapes survive a 15% noise floor and values do not. A reader who wants the width in cycles needs a machine with a PMU or a clock-domain trick, and the speculation concept is explicit about which claims each of those buys.