AI Grimoire

Miscellany

The shelf for what the other seven take for granted. Nothing here is a technique — these are the properties of the substrate that every technique is built on, and they earn a place because they are cross-cutting rather than because they are minor. A quantisation scheme, a training recipe and a serving stack all rest on the same floating-point behaviour, and none of them is the right place to explain it.

Three entries across two branches, none of them stubs.

Numerics

numerics/ ›

what the arithmetic actually does, as distinct from what the maths says

08.1
Floating-Point Formatsstandard
What bf16, fp16, fp8 and fp4 each give up, and which failure follows.
08.2
Nondeterminismstandard
Why the same prompt, the same weights and the same seed give different logits.

Conventions

conventions/ ›

the arbitrary choices that are not arbitrary once weights exist

08.3
Tensor Layout Conventionscommon
Axis order, pairing schemes and epsilon placement: arbitrary until weights exist.