{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4da04da6-64c1-5668-aa68-b5002b93b589","task_key":"all--test--4da04da6-64c1-5668-aa68-b5002b93b589","task_revision_id":"1","upstream_id":"","short_description":"Why were the elite of state societies so conspicuous in their consumption?","config":"all","split":"test","body":"{\"choices\":\"['Democracy and egalitarian states had not yet evolved.', 'A monetary value was not placed on material possessions, making things effectively worthless.', 'The elite had not yet been introduced to capitalism and so wasted their wealth on monuments and possessions.', 'Monuments and possessions were used as symbols to reinforce the inequalities on which state societies are based.']\",\"question\":\"Why were the elite of state societies so conspicuous in their consumption?\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}