{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"95075dfa-a2ac-5226-8486-5d55f1d345dd","task_key":"all--test--95075dfa-a2ac-5226-8486-5d55f1d345dd","task_revision_id":"1","upstream_id":"","short_description":"What do we know about the people who built Stonehenge based on the analysis of…","config":"all","split":"test","body":"{\"choices\":\"['They raised livestock at the site, including cattle, sheep, and goats.', 'They brought animals to the site from outside of the region.', 'They had no domesticated animals but hunted wild animals for meat.', 'They hunted antelope, gazelle, and bear.']\",\"question\":\"What do we know about the people who built Stonehenge based on the analysis of animal teeth from a nearby village?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}