AI Grimoire
08 — siblings
numerics/
conventions/

Miscellany / numerics

Numerics

Every equation in this book is written over the reals and executed over a few thousand representable values. Most of the time the gap does not matter. The entries here are the cases where it does — where the format decides whether a run converges, and where the order of a reduction decides whether two identical prompts produce identical logits.

2 entries.

Entries

08.01.1
Floating-Point Formatsstandard
What bf16, fp16, fp8 and fp4 each give up, and which failure follows.
—
08.01.2
Nondeterminismstandard
Why the same prompt, the same weights and the same seed give different logits.
—