Grimoire
Sheet
pathfine-tuning/peft
difficultyintermediate
timeO(d)
described2022
revised7w ago

Adapter Merging

Weight deltas from separately fine-tuned models can be added, negated and interpolated. Why that works at all is not settled.

Status

This entry is a stub. The maths has not been rederived by hand, or the implementation has not been run on real tensors, so it is recorded here as an open question rather than an answer.

Read what follows as a pointer to the sources, not as a settled account.

Notes

Write τt=θtθ0\tau_t = \theta_t - \theta_0 for the delta a fine-tune produced. Task arithmetic observes that θ0+tλtτt\theta_0 + \sum_t \lambda_t \tau_t often behaves like a model trained on the union of those tasks, and that negating a τ\tau subtracts the behaviour. TIES and DARE add conflict resolution: trim small entries, elect a sign per coordinate, and average only the agreeing deltas.

The reason this is a stub is that the explanation is not settled. Linear mode connectivity between fine-tunes of a shared base is the usual account, but it is a description of when merging works rather than a prediction, and the failure cases have not been rederived here.

Related
References
[1]Ilharco et al. — Editing Models with Task Arithmetic (2022)arXiv:2212.04089
[2]Yadav et al. — TIES-Merging: Resolving Interference When Merging Models (2023)arXiv:2306.01708