Reasoning / Knowledge
MMLU
mmlu
MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.
- Metric
- accuracy (fraction of questions answered correctly, usually reported as a macro average over the 57 subjects)
- Size
- 14,015 unique problems imported from the all / test selection of 14,042 rows; 27 source rows repeat an earlier question exactly and are recorded as variants. The 57 subject configurations overlap this selection and are not imported separately.
- License
- mit (declared on the Hugging Face dataset card)
- Reference links
- 4
Repository: Official repository