Crosscoders
One dictionary, many read and write sites. Fitting across layers exposes features that persist through depth; fitting across two checkpoints exposes what fine-tuning added.
Standing
PromisingPromising and actively moving. The results are real but narrow — one lab, one model family, or one benchmark suite — and the picture may look different in a year.
judged as of 2026-09 · what the labels mean
Status
This entry is a stub. The maths has not been rederived by hand, or the implementation has not been run on real tensors, so it is recorded here as an open question rather than an answer.
Read what follows as a pointer to the sources, not as a settled account.
Notes
An ordinary SAE reads one layer and writes back to it. A crosscoder reads a set of layers at once and reconstructs all of them from a shared dictionary, so a feature that persists across depth is represented once rather than refitted per layer.
Point the same machinery at two checkpoints instead of two layers and it becomes model diffing: latents that reconstruct only the fine-tuned model’s activations are, in principle, what the fine-tune added.
Stub. The claim that a shared dictionary identifies “the same” feature across sites is doing real work and has not been checked here.