Attribution Patching

Patching every component costs one forward pass each. A first-order Taylor expansion estimates all of them from two forward passes and one backward pass.

[causal][circuits]

Theory

Linearise the metric around the corrupted activation. The estimated effect of replacing $a$ with $a'$ is then the inner product of the gradient at $a$ with the difference — and the gradient for every component comes from a single backward pass.

\mathrm{effect}(a \rightarrow a') \approx (a' - a)^{\top} \frac{\partial M}{\partial a}

The approximation is poor precisely where the model is most interesting: at saturated softmaxes, at layer-norm scale changes, and anywhere the true effect is large. Integrated gradients along the path from $a$ to $a'$ recovers most of the lost fidelity for a handful of extra passes, and is the right default when the shortlist matters more than the sweep.

activation-patching path-patching sparse-autoencoders

References

[1]Nanda — Attribution Patching: Activation Patching At Industrial Scale (2023)neelnanda.io

[2]Syed et al. — Attribution Patching Outperforms Automated Circuit Discovery (2023)arXiv:2310.10348