Path Patching

Activation patching changes a component’s output for every downstream reader at once. Path patching changes it for one reader only, which is what a claim about an edge in a circuit actually requires.

[causal][circuits]

Theory

Because the residual stream is a sum, a component’s output reaches every later block. Patching it measures the total effect along all paths simultaneously. To isolate an edge $A \rightarrow B$ you must present the patched value of $A$ to $B$ alone, while every other consumer of $A$ continues to see the original.

\mathrm{effect}(A \rightarrow B) = M\bigl( \mathrm{do}(\mathrm{input}_B(A) \leftarrow a') \bigr) - M(\cdot)

Implementation requires three runs: a clean run, a corrupt run, and a run in which $B$ is recomputed from a residual stream that has had $A$ ‘s contribution swapped, with everything downstream of $B$ then freely recomputed. Doing it for all pairs is quadratic in components, which is why attribution methods exist.

activation-patching attribution-patching sparse-autoencoders

References

[1]Wang et al. — Interpretability in the Wild: a Circuit for Indirect Object Identification (2022)arXiv:2211.00593

[2]Goldowsky-Dill et al. — Localizing Model Behavior with Path Patching (2023)arXiv:2304.05969