Benchmark
A standardized test used to measure and compare how well different AI models perform on specific tasks.
01 Definition
A benchmark is a fixed set of test questions or tasks (like math problems, coding challenges, or reading comprehension) used to score AI models consistently, so their capabilities can be compared against each other on a level playing field.
02 Why It Matters
Benchmark scores are how AI companies market model quality ('our model scores X% on this benchmark'), but they don't always reflect real-world usefulness for your specific task — treat them as one signal, not the whole picture.
03 Example
MMLU and HumanEval are common benchmarks for general knowledge and coding ability, respectively.