AI Grimoire

Miscellany

The shelf for what the other seven take for granted. Nothing here is a technique — these are the properties of the substrate that every technique is built on, and they earn a place because they are cross-cutting rather than because they are minor. A quantisation scheme, a training recipe and a serving stack all rest on the same floating-point behaviour, and none of them is the right place to explain it.

Three entries across two branches, none of them stubs.

Numerics

numerics/ ›

what the arithmetic actually does, as distinct from what the maths says

08.1
Floating-Point Formatsstandard
What bf16, fp16, fp8 and fp4 each give up, and which failure follows.
—
08.2
Nondeterminismstandard
Why the same prompt, the same weights and the same seed give different logits.
—

Conventions

conventions/ ›

the arbitrary choices that are not arbitrary once weights exist

08.3
Tensor Layout Conventionscommon
Axis order, pairing schemes and epsilon placement: arbitrary until weights exist.
—