Back to course

Annotations

20 minadvancedModule 3: The Attribute MechanismContinues from jvm-signatures
20 min · Module 3: The Attribute Mechanism · Algorithms

Why This Matters

An annotation is metadata attached to a declaration and read by reflection. The interesting question is not what that means but how a container format encodes a value whose type is itself a name, because that is a recursion the format has to break somewhere.

It breaks it in the most economical way available. An annotation value can be a primitive, a String, an enum constant, a class literal, another annotation, or an array of any of those — and the encoding of the value is always a constant pool index, never an inline value. So @Marker(num = 2) does not put the bytes for 2 in the attribute; it puts an index to a CONSTANT_Integer that holds 2, and that entry is shared with every other use of 2 in the file.

And the type tag that says which kind it is is the character itself. 0x49 is I for int, 0x73 is s for String, 0x5A is Z for boolean. A number would have done; a character is what a human reading a hex dump actually wants, and it is the one place in the format where the encoding is chosen for legibility over compactness.

A Simple Model

The attribute's body, and the shape of a value inside it:

RuntimeVisibleAnnotations: u2 number_of_annotations then that many: u2 type_index a Utf8: the annotation's DESCRIPTOR, e.g. "LMarker;" -- note the L and ; u2 number_of_element_value_pairs then that many: u2 element_name_index a Utf8: the member's name element_value value element_value: u1 tag THE CHARACTER -- see below u2 const_value_index ALWAYS a constant pool index

That last line is the whole concept. The tag says how to interpret the pool entry the index points at, and every value is a reference. The types and their tags:

  tag  char  type            points at
    0x42  B     byte             CONSTANT_Integer
    0x43  C     char             CONSTANT_Integer
    0x44  D     double           CONSTANT_Double
    0x46  F     float            CONSTANT_Float
    0x49  I     int              CONSTANT_Integer
    0x4A  J     long             CONSTANT_Long
    0x53  S     short            CONSTANT_Integer
    0x5A  Z     boolean          CONSTANT_Integer
    0x73  s     String           CONSTANT_Utf8
    0x65  e     enum             CONSTANT_Utf8 -- "EnumType.fieldName"
    0x63  c     class            CONSTANT_Utf8 -- a descriptor
    0x40  @     annotation       a nested annotation, inline

Three things fall out of that table, and each is a trap.

The primitives share three pool tags.byte, char, short, int and boolean are all CONSTANT_Integer — the tag byte is the only thing distinguishing them, and it is a byte that was kept purely so a hex dump is readable. The pool entry cannot tell you which of the five it is.

s is lowercase and S is uppercase. That is the single case-difference in the whole format, it is one bit, and it is the difference between String and short. A reader that uppercases its tags, or that compares them case-insensitively, will mis-read a string value as a short and then misinterpret the following bytes.

And an enum constant is a CONSTANT_Utf8 holding "EnumType.fieldName" — a dotted pair in one string, not a class reference and a name index. A class literal is a CONSTANT_Utf8 holding a descriptor, so Foo.class is the string "LFoo;" and not a CONSTANT_Class index. Both are a deliberate flattening: an enum value and a class value have no identity to the verifier, so they get no pool entry of their own.

The Actual Detail

A complete annotation, all eleven bytes of it, from the class-level @Marker("hello"):

  00 01        ONE annotation

  00 0e        type_index = 14, and entry 14 is
               the Utf8 "LMarker;"     <- a DESCRIPTOR, not a Class

  00 01        ONE element-value pair

  00 0f        name_index = 15, and entry 15 is
               the Utf8 "value"

  73           tag 0x73 = 's' = String

  00 16        const_value_index = 22, and entry 22 is
               the Utf8 "hello"

  total: 11 bytes, and the value "hello" is stored in the POOL,
  not in the attribute.

Now two annotations with two pairs each and three different value types, to see the machinery repeat:

  00 02 00 0e 00 02 00 0f 73 00 10 00 11 49 00 12
  |  |  |     |     |     |  |     |  |  |
  |  |  |     |     |     |  |     |  |  +-- #18 = Integer 2
  |  |  |     |     |     |  |     |  +----- #17 = "num", tag 'I' = int
  |  |  |     |     |     |  |     +-------- #16 = Utf8 "m"
  |  |  |     |     |     |  +-------------- tag 's' = String
  |  |  |     |     |     +----------------- #15 = "value"
  |  |  |     |     +----------------------- 2 pairs
  |  |  |     +----------------------------- #14 = "LMarker;"
  |  |  +--------------------------------------- 2 annotations
  |  +-------------------------------------------- ONE annotation
  +------------------------------------------------ TWO annotations

  00 13 00 00
  #19 = "Ljava/lang/Deprecated;" with ZERO pairs

And the parser output, which agrees with the bytes:

  method go: 20 bytes
    [0] type = LMarker;
            value   tag=0x73 's' = String  -> #16
            num     tag=0x49 'I' = int     -> #18
    [1] type = Ljava/lang/Deprecated;
            zero pairs
    consumed 20 of 20   OK

Twenty bytes, and not one of them is a value. Four pool indices carry the four pieces of information and the rest is shape. That is the encoding's whole character, and it is the same economy the constant pool provides everywhere else — a name used twice is stored once.

The mistake this course made, and what it cost

Decoding that middle pair, the first analysis assumed the int payload was four bytes inline and read 00 12 00 13 as the value 1179667. The source said num = 2.

The way that was caught is worth recording, because the arithmetic did not merely look wrong — it did not close. Eleven bytes of attribute plus ten more had to account for exactly twenty, and a four-byte payload left two bytes over, which then had to be the start of the next annotation. It was not, because type_index = 0 is the reserved constant pool entry and no valid annotation can name it. A parse that does not balance is a parse that is wrong, and the imbalance is louder evidence than a surprising number.

The rule is a uniform two-byte payload, and the reason is arithmetic rather than tradition. Every value is a pool index, so int costs two bytes pointing at an eight-or-four-byte entry rather than four bytes inline — and long costs two bytes pointing at a two-slot entry rather than eight inline, where the two-slot rule from the constant pool would have cost nine with padding. A uniform u2 is never larger than any inline encoding and is sometimes half the size, and it comes with deduplication for free. That is the whole argument, and it is the same one that put the constant pool at the centre of the format.

Why there are two annotation attributes

A class carries RuntimeVisibleAnnotations and RuntimeInvisibleAnnotations, and the distinction is a retention policy — @Retention(RetentionPolicy.CLASS) versus RUNTIME — enforced by the compiler choosing which attribute to emit. Both are in the file; only one is read by reflection.

So RetentionPolicy.SOURCE annotations are not in the class file at all, CLASS ones are in a file no runtime API will show you, and RUNTIME ones are in the attribute reflection reads. Three policies, two attributes, and one of them is the absence of the annotation — which is the only case in the whole format where the way to store a declaration is not to store it.

And the class-level example shows the naming convention for the whole family: RuntimeVisibleAnnotations, RuntimeInvisibleAnnotations, RuntimeVisibleParameterAnnotations, RuntimeVisibleTypeAnnotations, and the Invisible and Type variants of each. Four orthogonal axes — visibility, parameter versus declaration, type versus declaration, and a Repeatable container — multiplied out into a dozen attribute names, all identical in body shape. It is a small illustration of a cost the escape hatch does not avoid: a new axis would mean a new attribute name, not a new structure.

A Real Example

Why the encoding puts everything in the pool, told through the two things that break when it does not. Both are from tools that exist and both are silent.

PROBLEM 1: arrays and nested annotations have no pool tag@Anno with ints = an array 1, 2, 3 what is the tag? There isn't one. The specification says a value with tag '[' is... not in the list. Arrays are encoded as the ANNOTATION COUNT mechanism: an annotation with n values has n+1 pairs, where the LAST pair has an empty name and the array's values. @Anno(nested = @Inner("x")) the tag is '@' and the value is a WHOLE NESTED ANNOTATION, inline, recursively. No pool entry. so two of the eight value kinds are NOT pool references, and a reader that assumed every value is a u2 index will mis-parse both.

That is the real complication, and it is worth being precise about. Six of the eight kinds are a one-byte tag and a u2 index. The array form reuses the pair list with an empty name, and the annotation form embeds a whole annotation recursively. So element_value is not a fixed-size record, and a reader cannot walk an annotation attribute without branching on the tag — which is the first genuinely variable-length structure inside an attribute body in this format.

And the second problem is the one a rewriter hits. Because every value is a pool index, renaming anything means rewriting the pool, not the attribute. A tool that renames a class from Foo to Bar must update the CONSTANT_Class entry, the CONSTANT_Utf8 for its name, every descriptor mentioning it, and every signature — and if an annotation holds a class literal, the value is a CONSTANT_Utf8 holding a descriptor string, which is not reachable by following the CONSTANT_Class index. It is a string that looks like a name, in a pool entry that nothing else points at.

Compare a format where a class reference is always a reference: in a COFF relocation the target is a symbol index, and a rewriter follows it. Here, a class literal in an annotation is a string that must be parsed to discover it names a class, and a rewriter that only follows indices will miss it entirely. Flattening a type into a string buys uniformity for the value encoding and costs the ability to find it by reference — and that is a genuine trade, not a free win, and it is the same trade as the signature's inline type arguments making.

One last thing about the tag bytes, because it is the sort of detail that separates a reader from a decoder. They are the ASCII codes of the type's first letter, case-sensitive, with one exception that is not a letter at all: @ for an annotation, which is 0x40 and is a symbol rather than a character. So nine of ten tags are letters, the tenth is punctuation, and s is lowercase among eight uppercase ones. A tool that wants to print a tag should map it through a table rather than transform it, because chr(0x73) is already the right answer and any case-folding breaks it.

Try It Yourself

$ javap -v -p out/Anno.class
$ python3 courses/jvm/assets/samples/crosscheck.py classes/Anno.class

Check Your Understanding

Without looking back: an annotation's element-value pair for num has the name "num", the tag byte 0x49, and the bytes 00 12. The source says num = 2. What is the tag, what is #18, and how many bytes of the attribute does the value 2 itself occupy?

Apply It

You are writing a tool that reads annotations. On one class it reports @Marker(value="m", num=0) where the source says num=2, and it consumes exactly the right number of bytes. On a second class it desynchronises and reports nonsense for every annotation after the first. The first class has @Marker(value="m") and a deprecated method; the second has @Marker(num=2) and an array-valued annotation. What are the two defects, and what is the general habit for parsing a value whose type is a tag byte?

The general habit: a tag byte is a type discriminator and everything after it depends on it. Resolve the tag first, then the payload — and when a value can be variable-length, check that the parse balances before trusting any of the values you read.

Connect

Annotations are the format's most conventional corner, and the comparisons are instructive precisely because everything else about them is unremarkable. A DWARF attribute is a tag, a form, and a value — the same three-part shape — and DWARF's DW_FORM is a genuine enumerated tag byte rather than a character. DWARF chose numbers because it was specified for machines to read and for people to write generators, and the JVM chose characters because a class file is overwhelmingly read with a hex dump. Neither is better; the choice tracks who the reader is.

The more interesting comparison is what a DWARF attribute value can be. DW_FORM_ref_addr, DW_FORM_strx and DW_FORM_addrx are all indirect forms that point at a string, a line table or the address table rather than carrying the value — exactly the choice the annotation encoding makes, for exactly the reason: the value is bigger than the form, or it is shared, or both. And DWARF went further, adding implicit forms where the value is computed from the object's address and appears nowhere in the file, which is a step further in the same direction and a good illustration of how far the trend goes once a format commits to referencing rather than embedding.

The pooling choice also lands where the rest of this course has been going. A PE load config's cookie fields are inline because they are small, unique, and fixed; a COFF symbol's name is an offset into a string table because names are neither small nor unique. Annotation values are small but not unique — num = 2 appears in thousands of annotations — and that is the case a string table exists to serve. The format picked the table because the reuse was the dominant consideration, and picked it uniformly, so that a reader needs one rule rather than one per value type.

The trade that uniformity costs is worth keeping in view, because it is the same trade the signature grammar makes. Inline a value and it becomes findable by position; reference it and it becomes shared but only reachable by an index. A class literal in an annotation is the case that bites: it is the string "LFoo;" in a pool entry nothing else points at, so a rewriter following indices misses it. The same flattening in the signature grammar means a repeated type argument is written out twice rather than shared. Both are grammars that buy a uniform, simple structure by giving up reachability, and both are reasonable, and a tool that walks a class file by reference rather than by parsing has to know which of the two it is looking at.

That closes Module 3. The mechanism, the names, the type information and the metadata are all attributes now — which is the point of the module, and the reason the format needed no new layout in twenty years. Module 4 takes the three shapes the language has added most recently, and they are the three that have needed the most from the mechanism: a record component that carries its own attributes, a permitted subclass that is a bare list of indices, and an enum that is a class shape the language has to synthesise wholesale.