07 — siblings

metrics/

Evaluation / metrics

Metrics

Evaluation is where the field is least rigorous and most confident. These entries cover the intrinsic measures, the pairwise-comparison models borrowed from chess, and the practice of asking a language model to grade — along with the biases each of them carries in.

3 entries.

Entries

Elo / Bradley-Terry Scoringcommon

Latent-strength model fitted to pairwise preference outcomes.

LLM-as-Judgecommon

Using a model to score model outputs, and the biases that come with it.

Perplexitycommon

Exponentiated mean negative log-likelihood over a held-out corpus.