Grimoire
Sheet
pathinterpretability/features
difficultyadvanced
timeO(L·d·m)
described2024
revised3w ago

Crosscoders

One dictionary, many read and write sites. Fitting across layers exposes features that persist through depth; fitting across two checkpoints exposes what fine-tuning added.

Status

This entry is a stub. The maths has not been rederived by hand, or the implementation has not been run on real tensors, so it is recorded here as an open question rather than an answer.

Read what follows as a pointer to the sources, not as a settled account.

Notes

An ordinary SAE reads one layer and writes back to it. A crosscoder reads a set of layers at once and reconstructs all of them from a shared dictionary, so a feature that persists across depth is represented once rather than refitted per layer.

Point the same machinery at two checkpoints instead of two layers and it becomes model diffing: latents that reconstruct only the fine-tuned model’s activations are, in principle, what the fine-tune added.

Stub. The claim that a shared dictionary identifies “the same” feature across sites is doing real work and has not been checked here.

Related
References
[1]Lindsey et al. — Sparse Crosscoders for Cross-Layer Features and Model Diffing (2024)transformer-circuits