{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c56fb5a0-d85b-5c7d-800b-ba5ff7b066cc","task_key":"all--test--c56fb5a0-d85b-5c7d-800b-ba5ff7b066cc","task_revision_id":"1","upstream_id":"","short_description":"During a deer-hunting season open to rifle hunters, a hunter saw a deer in the…","config":"all","split":"test","body":"{\"choices\":\"['No, because the hunter did not intend to shoot the hiker.', 'No, because the hunter did not make direct physical contact with the hiker.', \\\"Yes, because the bullet from the hunter's rifle made direct physical contact with the hiker.\\\", 'Yes, because the hunter intentionally shot the rifle.']\",\"question\":\"During a deer-hunting season open to rifle hunters, a hunter saw a deer in the forest. He shot his rifle at the deer, hoping to hit and kill it. Instead, he hit and injured a hiker. The hunter had not realized that the hiker was there. Does the injured hiker have an actionable battery claim against the hunter?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}