RISC-V Atomics and the Vector Extension

fence, and an Ordering Model Defined in Terms of It

26 minadvancedModule 1: The Reservation, and the Fence That Defines the ModelContinues from rv-amo
26 min · Concept 2 of 4 · module: atomic · RISC-V Atomics and the Vector Extension

Three models, and only one of them makes the fence the primitive

The finding on this page is not a number. It is a shape. Three architectures in this collection have a memory-ordering model, and they are not three degrees of one idea:

A baseline-and-escape model is a claim about what happens when you do nothing. An access-mode model is a claim about what each instruction means. A predecessor/successor model is a claim about a relation between two sets. IT IS A DIFFERENCE OF FORM, and the test is a reduction: a reader who knows what one fence does on RISC-V can predict what every other ordering operation does, because every other one of them is a fence with a part removed. There is no such reduction available in the other two directions, and that asymmetry is the whole point.

Which is why concept 1 ends where this one begins. The two bits on an AMO are not a fourth thing to learn; they are the difference between one fence and another, recorded in the instruction instead of written out.

Four fields, fifteen spellings, and one hole

MEASURED-ON-BYTES. fence carries fm at inst[31:28], a four-bit pred at [27:24], a four-bit succ at [23:20], and then rs1 and rd — which are reserved and zero. Here is the whole table, with the spelling rebuilt from the decoded bits rather than copied from the reader:

$ python3 rvat.py --run | sed -n '/THE FOUR FIELDS, read/,+8p'
  word        the spelling   fm      pred       succ       rs1  rd  funct3
  0x0ff0000f  fence          0b0000  iorw = 15  iorw = 15  0    0   0
  0x0ff0000f  fence          0b0000  iorw = 15  iorw = 15  0    0   0
  0x0220000f  fence r, r     0b0000  r = 2      r = 2      0    0   0
  0x0110000f  fence w, w     0b0000  w = 1      w = 1      0    0   0
  0x0330000f  fence rw, rw   0b0000  rw = 3     rw = 3     0    0   0
  0x0230000f  fence r, rw    0b0000  r = 2      rw = 3     0    0   0
  0x0320000f  fence rw, r    0b0000  rw = 3     r = 2      0    0   0
  0x0f20000f  fence iorw, r  0b0000  iorw = 15  r = 2      0    0   0
                

And that rebuild is not a style choice. If the spelling column were copied from the same reader the encoding came from, a row would be right twice or wrong once, and the table could not carry a measurement at all. Printing both columns independently is what lets the table be checked.

Two reserved fields, and the manual is why they are zero

Every one of those words has rs1 = 0 and rd = 0. It is tempting to read that as a property of the assembler — and the artifact retracted exactly that reading. The manual reserves both fields for finer-grain fences in future extensions, says base implementations shall ignore them, and says standard software shall zero them:

Two reserved fields in an instruction whose other two fields carry the whole model is the tell that the encoding was designed for a future nobody has built yet. It is a small observation and it generalises: when you read an encoding and find fields that are always zero, ask whether a document requires the zero or whether the toolchain merely emits it. The first is an architecture; the second is a habit, and habits are not portable.

Three instructions, one opcode

$ python3 rvat.py --run | grep -A 1 'ordering fences at fm'
  35 ordering fences at fm = 0000, 2 fence.tso at fm = 1000, and
  2 fence.i at funct3 = 1.  THREE INSTRUCTIONS, ONE OPCODE,
                

Three instructions share opcode 0x0f, and the ordering is by funct3 and by fm rather than by the opcode alone. So a decoder that dispatches on the opcode gets the wrong one of the three — and this course had to prepend its own model to the borrowed decoder for exactly that reason, because the inherited one reads fm and then calls every word fence. That is a defect in a sibling course’s code, published rather than quietly patched, and it is retraction R3.

Fifteen spellings, and the sixteenth is a hole

“The sets are named and not arbitrary” sounds like a claim about a document. It is a claim about the assembler, so the artifact sweeps it: every non-empty subset of {i, o, r, w}, one assembler call each, and the four-bit code each one gets.

$ python3 rvat.py --run | sed -n '/THE FIFTEEN SPELLINGS/,+19p'
  bits    value  the name  an instruction the assembler emitted
  0b0001  1      w         fence
  0b0010  2      r         fence
  0b0011  3      rw        fence
  0b0100  4      o         fence
  0b0101  5      ow        fence
  ... through ...
  0b1111  15     iorw      fence

  15 distinct predecessor codes reached, of the 16 the field can hold.
                

Fifteen of sixteen. And the sixteenth is the empty set, which the assembler will not spell at all — fence with an empty operand list is refused with “too few operands for instruction”. So the field has sixteen values, fifteen of them have names, and one of them is not reachable by name. That is why the table prints the sixteenth as a hole rather than a gap: a reader who has not seen the empty value cannot know whether it is reserved or merely unused, and those are different things to a decoder.

And the four letters are not interchangeable, which is the part that makes the field a set rather than a mask. i is device input, o is device output, r is a memory read, w is a memory write — and the manual separates the two domains deliberately: “the address space is divided by the execution environment into memory and I/O domains, and the FENCE instruction provides options to order accesses to one or both of these two address domains.” So fence r, w and fence io, io are not two spellings of one thing in different amounts. They order different domains. A four-bit field whose four bits are two pairs from two independent axes is not a four-state enumeration; it is two two-state enumerations, and a decoder that treats it as one will eventually have to say “iorw” when it means “both domains, everything”.

What the compiler chose, and where it stopped

The model being definitional is a fact about the ISA. The model being used is a fact about clang 21.1.8. A course that conflates them is making a claim about one from evidence for the other, so the table is here in full:

$ python3 rvat.py --run | sed -n '/AND WHAT THE COMPILER CHOSE/,+8p'
  function         instr  fences  what it emitted
  .Lpcrel_hi0      3      0       NONE
  fence_acqrel     2      1       fence.tso (bare)
  fence_acquire    2      1       fence (r, rw)
  fence_consume    2      1       fence (r, rw)
  fence_relaxed    1      0       NONE
  fence_release    2      1       fence (rw, w)
  fence_seqcst     2      1       fence (rw, rw)
  load_acquire     3      1       fence (r, rw)
  load_seqcst      4      2       fence (rw, rw); fence (r, rw)
                

Three rows in that table are worth reading twice, and all three are the compiler translating rather than choosing.

CONSUME and ACQUIRE are the same four bits. The C11 distinction between them is not representable in this encoding and the compiler does not pretend otherwise — it emits one word for both. A language feature the hardware cannot express shows up here as a diagnostic-free silent merge, and finding it costs one table.

ACQ_REL emits fence.tso and SEQ_CST emits fence rw, rw. That first one is the surprise: an acquire-release fence, whose spelling names a barrier, compiles to an instruction that is not fence at all. It is fm = 1000 — a different instruction that shares an opcode. The mnemonic you grep for is not always the mnemonic that gets emitted, and a build that greps assembly for “fence” will miss half the fences.

An acquire load and a sequentially consistent load are the same lw — four bytes, one opcode, byte for byte — and the difference is discharged by a fence beside it. On AArch64 the same two C11 operations are ldar and ldar, literally the same instruction; on x86-64 both are a plain mov and the difference is a mfence or nothing at all. RISC-V puts the ordering in a separate instruction because its model says ordering is a relation between two sets and not a property of an access. And measured: the something else is a second fence. load_seqcst emits fence rw, rw before the load and fence r, rw after it — two fences for one load, because the extra constraint a seq_cst load carries is on its relation to other operations, and that is discharged by ordering the neighbours rather than by changing the load.

The row that is not a fence at all, and the one that is not fence

fence_relaxed is a bare ret. A relaxed fence is not a no-op instruction; it is the absence of one. A function that consists of a single return is a correct compilation of a function that asks for nothing, and the encoding for “no ordering” is no instruction. Eight of the twenty-one functions in that table emitted no fence at all.

And then there is fence.tso, which is where the page has to stop, and the stopping is the content.

This is where the page has to stop, and the stopping is the content. fence.tso is fm = 1000 with pred = RW and succ = RW — the same sets as fence rw, rw, differing only in the fm field, which the manual calls reserved for future use. The natural guess is that it is a weaker barrier and that emitting it is a request. The manual says something stronger, and the direction is the interesting part:

    rv32-unpriv, chapter 2.7:
      "Because FENCE RW,RW IMPOSES A SUPERSET OF THE ORDERINGS THAT
       FENCE.TSO IMPOSES, IT IS CORRECT TO IGNORE THE fm FIELD AND
       IMPLEMENT FENCE.TSO AS FENCE RW,RW."
                

So the instruction is a request for a weaker barrier than the one it is spelled like, and a base implementation is explicitly permitted to give the stronger one. Which means “emit a fence.tso” is not portable advice and also not a weaker barrier, and the only honest thing to say is both.

And this course cannot tell you which one a machine does. It is measured that the compiler emits it for __ATOMIC_ACQ_REL, that it is fm = 1000 with pred = RW and succ = RW, and that the manual permits the stronger implementation. Whether the machine gives the weaker barrier or the stronger one is a property of silicon, and there is no RISC-V silicon on this host. That is limit 6, and it is quoted beside the measurement rather than in a footnote.

Try it

  1. Count the fence strength you did not ask for.(Compile __atomic_thread_fence(__ATOMIC_ACQUIRE) and read the disassembly. Then compare with the lr.aq form of an acquire load. The first is a relation between two sets; the second is the same relation with a part deleted. Now ask which of the two is cheaper to reason about in your own code, and notice that the answer is not the answer to which is cheaper on hardware.)
  2. Find the hole.(There are sixteen values of pred and fifteen names. Try to get the assembler to emit the sixteenth — fence with no sets at all. Read the diagnostic. Now write a decoder that has no case for it and consider what it will print for the zeroed pred of a fence.i, which is a different instruction entirely.)
  3. Grep your own build.(Search a disassembly for fence. Count the instructions you find. Now search for fence.tso and count again. The difference is real: fm = 1000 is a separate instruction that shares the opcode and will not appear in a mnemonic grep for “fence” in some tools and will in others. Which is exactly why the page prints the three-instruction-one-opcode count.)

Connect

What ordering means — what a race is, what happens when two harts see stores in different orders — is in smp-ordering, and it is not re-taught here. The same model on the two machines this collection can run is x86-order, where the baseline is TSO and mfence is the escape, and a64-order, where the ordering rides on the access as a mode and dmb is the coarse one. rv-noflags is why the whole model has to be spelled out in instructions at all: an architecture with no condition codes has nowhere to put a result except a register, so every ordering operation is a real instruction with a name.

Next: vsetvli and the Four Fields It Carries, which is the same kind of idea in a different place — a fixed 32-bit word that describes a vector whose length is not in it.