Standing note
Kept open, revised often.
32 entries, 6 stubs.
No prose written by a model.
32 entries, 6 stubs.
No prose written by a model.
The AI Engineer's Grimoire
A field reference for how these systems actually work: attention and its cheaper cousins, training dynamics that fail quietly, inference kernels, the interpretability toolkit, and the attack surface that follows all of it.
Each entry begins with the maths and ends with something you can run. Nothing is explained twice.
Query log
$ grimoire search
32 entries scanned · 3 matches · 7ms
name Scaled Dot-Product Attention
path architectures/attention/scaled-dot-product-attention
classO(n²·d) time /O(n²) memory
softmax( Q KT ⁄ √dk ) V
open with grimoire cat <path>
Categories
01
Architectures — attention variants, position schemes, normalisation, sparse blocks
502Training — optimisers, schedules, sharding, RL objectives, stability failures
503Fine-tuning — PEFT families, adapter merging, data curation, catastrophic drift
304Inference — kernels, KV-cache layouts, quantisation, speculative decoding
505Interpretability — circuits, SAEs, causal interventions, probing, feature geometry
706Adversarial ML — evasion, extraction, poisoning, privacy leakage, agent hijacks
407Evaluation — metrics, preference aggregation, contamination, statistical power
3Recent revisions
Scaled Dot-Product Attention
Content-based retrieval over key–value pairs with 1/√d_k logit scaling.
Architectures · revised 4d ago
O(n²·d)
Sparse Autoencoders
Overcomplete dictionary learning on activations for monosemantic features.
Interpretability · revised 4d ago
O(d·m)
FlashAttention
Tiled, IO-aware attention kernel; recomputes softmax statistics online.
Inference · revised 6d ago
O(n²·d)
Speculative Decoding
Draft model proposes tokens; target model verifies in one pass.
Inference · revised 6d ago
O(γ·n)