With corpus access, contamination is a string-matching problem: mark a test example dirty if a sufficiently long -gram of it appears in training. Thirteen tokens is the common threshold, chosen so that natural collisions are rare in English but paraphrase is not caught.
Without corpus access — the normal case — the tests are behavioural. The cleanest is an exchangeability test: under no contamination, a model’s log-likelihood should be invariant to the order of a benchmark’s multiple-choice options or of its examples. A significant preference for the canonical ordering is evidence of memorisation, since only a memorised copy carries the ordering.
Canaries are the prospective fix: publish a benchmark with an embedded random string and a request to exclude it, then later ask the model to complete the string. This detects only honest contamination, which is nevertheless most of it.
The strongest available evidence is usually a held-out replication: build a fresh test set by the same procedure and compare. A model that drops several points on the replication was reading, not reasoning.