Grimoire
Sheet
pathevaluation/data
difficultyintermediate
timeO(N)
described2020
revised6w ago

Contamination Detection

Assume the benchmark is in the training set until shown otherwise. The question is not whether contamination happened but how much of the score it explains.

Theory

With corpus access, contamination is a string-matching problem: mark a test example dirty if a sufficiently long nn-gram of it appears in training. Thirteen tokens is the common threshold, chosen so that natural collisions are rare in English but paraphrase is not caught.

Without corpus access — the normal case — the tests are behavioural. The cleanest is an exchangeability test: under no contamination, a model’s log-likelihood should be invariant to the order of a benchmark’s multiple-choice options or of its examples. A significant preference for the canonical ordering is evidence of memorisation, since only a memorised copy carries the ordering.

H0:    logp(canonical)  =d  logp(shuffled)H_0: \;\; \log p(\text{canonical}) \;\overset{d}{=}\; \log p(\text{shuffled})
eq. 1 — permutation test over orderingsOren et al. §3

Canaries are the prospective fix: publish a benchmark with an embedded random string and a request to exclude it, then later ask the model to complete the string. This detects only honest contamination, which is nevertheless most of it.

The strongest available evidence is usually a held-out replication: build a fresh test set by the same procedure and compare. A model that drops several points on the replication was reading, not reasoning.

Related
References
[1]Brown et al. — Language Models are Few-Shot Learners, §4 (2020)arXiv:2005.14165
[2]Oren et al. — Proving Test Set Contamination in Black Box Language Models (2023)arXiv:2310.17623
[3]Zhang et al. — A Careful Examination of Large Language Model Performance on Grade School Arithmetic (2024)arXiv:2405.00332