Filter by tag
Sort
Index
Every entry in the grimoire, including stubs. Filter in the margin, grep below.
/32/32
Activation Patching
Causal intervention swapping activations between clean and corrupt runs.
causalcircuits
Interpretability
O(L·n)
Adapter Mergingstub
Task arithmetic, TIES and DARE over independently trained deltas.
fine-tuningpeft
Fine-tuning
O(d)
Attribution Patching
First-order approximation to patching via a single backward pass.
causalcircuits
Interpretability
O(L·n)
Contamination Detection
n-gram and canary overlap tests between benchmark and pretraining data.
metricdata
Evaluation
O(N)
Crosscodersstub
Shared dictionaries fitted across layers or across model checkpoints.
featuressparse
Interpretability
O(L·d·m)
Direct Preference Optimisation
Closed-form preference objective that removes the reward model.
RLalignment
Training
O(d)
Elo / Bradley-Terry Scoring
Latent-strength model fitted to pairwise preference outcomes.
metricpreference
Evaluation
O(P)
Feature Absorptionstub
When one SAE latent swallows a more specific feature’s activation mass.
featuressparse
Interpretability
—
FlashAttention
Tiled, IO-aware attention kernel; recomputes softmax statistics online.
attentionkernelsmemory
Inference
O(n²·d)
GPTQ
Layer-wise second-order weight quantisation with Hessian-guided rounding.
quantisation
Inference
O(d³)
Gradient Checkpointing
Trade recompute for activation memory at segment boundaries.
memorytraining
Training
O(√L)
Logit Lens
Project intermediate residual states through the unembedding matrix.
probing
Interpretability
O(d·V)
LoRA
Low-rank additive update to frozen weight matrices during adaptation.
fine-tuningpeft
Fine-tuning
O(r·d)
Membership Inference
Distinguish training members from held-out data via loss calibration.
privacyattack
Adversarial ML
O(N)
Mixture-of-Experts Routing
Top-k gate over expert FFNs with auxiliary load-balancing loss.
sparsescaling
Architectures
O(k·d²)
Model Extraction
Query-only reconstruction of a victim model's decision function.
attackstealing
Adversarial ML
O(Q·d)
Multi-Query Attention
Single shared K/V head across query heads to shrink the decode cache.
attentionkv-cache
Inference
O(n·d)
Paged KV Cachestub
Virtual-memory paging applied to the attention cache.
kv-cachememory
Inference
O(1)
Path Patching
Restrict the intervention to a single computational path between components.
causalcircuits
Interpretability
O(L²·n)
Perplexity
Exponentiated mean negative log-likelihood over a held-out corpus.
metric
Evaluation
O(n)
Projected Gradient Descent Attack
Iterative L_p-bounded perturbation maximising task loss.
attackrobustness
Adversarial ML
O(k·d)
Prompt Injection Taxonomy
Direct, indirect and tool-mediated instruction hijack surfaces.
attackagents
Adversarial ML
—
Proximal Policy Optimisation
Clipped surrogate advantage objective with KL control to a reference.
RLalignment
Training
O(T·d)
QLoRA
4-bit NF4 base weights with paged optimiser state and LoRA adapters.
fine-tuningquantisation
Fine-tuning
O(r·d)
RMSNorm
Re-scaling without re-centring; LayerNorm minus the mean subtraction.
normalisationcore
Architectures
O(n·d)
Rotary Position Embedding
Relative position injected as a per-pair rotation in complex subspaces.
positionattention
Architectures
O(n·d)
Scaled Dot-Product Attention
Content-based retrieval over key–value pairs with 1/√d_k logit scaling.
attentiontransformercore
Architectures
O(n²·d)
Second-Order Preconditioningstub
Shampoo, Muon and the orthogonalised-update family.
optimisertraining
Training
O(d^1.5)
Sliding-Window Attentionstub
Local band mask; receptive field grows linearly with depth.
attentionsparse
Architectures
O(n·w·d)
Sparse Autoencoders
Overcomplete dictionary learning on activations for monosemantic features.
sparsefeatures
Interpretability
O(d·m)
Speculative Decoding
Draft model proposes tokens; target model verifies in one pass.
decodinglatency
Inference
O(γ·n)
ZeRO Sharding
Partition optimiser state, gradients and parameters across ranks.
distributedmemory
Training
O(d/N)
no matches — 0 of 32