{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd6cc182-e7e7-5c0d-9d40-72ad0520def7","task_key":"all--test--fd6cc182-e7e7-5c0d-9d40-72ad0520def7","task_revision_id":"1","upstream_id":"","short_description":"A 12-year-old girl and her friends were playing catch with a baseball in the…","config":"all","split":"test","body":"{\"choices\":\"['recover for assault only.', 'recover for battery only.', 'recover for assault and battery.', 'not recover.']\",\"question\":\"A 12-year-old girl and her friends were playing catch with a baseball in the middle of a field of grass in the park. Near the edge of the field, a woman was sitting in a beach chair and reading a book. The girl threw the ball too far, and it went over her friends' heads and flew toward the woman. Although the woman did not see the ball coming, it hit the straw hat that the woman was wearing, knocking it from her head. Although the woman was not touched by the ball, she was startled by the ball hitting her hat, and she fell from her chair and broke her arm. If the woman initiates a suit against the girl to recover damages for her broken arm, the woman will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}