Adapter Merging
Weight deltas from separately fine-tuned models can be added, negated and interpolated. Why that works at all is not settled.
Standing
Commonly usedEstablished and frequently the right choice, but competing with live alternatives rather than having settled the question.
judged as of 2026-09 · what the labels mean
Status
This entry is a stub. The maths has not been rederived by hand, or the implementation has not been run on real tensors, so it is recorded here as an open question rather than an answer.
Read what follows as a pointer to the sources, not as a settled account.
Notes
Write for the delta a fine-tune produced. Task arithmetic observes that often behaves like a model trained on the union of those tasks, and that negating a subtracts the behaviour. TIES and DARE add conflict resolution: trim small entries, elect a sign per coordinate, and average only the agreeing deltas.
The reason this is a stub is that the explanation is not settled. Linear mode connectivity between fine-tunes of a shared base is the usual account, but it is a description of when merging works rather than a prediction, and the failure cases have not been rederived here.