{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32d25e79-78e5-587d-b2fb-0967c319d788","task_key":"all--test--32d25e79-78e5-587d-b2fb-0967c319d788","task_revision_id":"1","upstream_id":"","short_description":"A child was playing with a ball in the front yard of his family's home. He…","config":"all","split":"test","body":"{\"choices\":\"['recover, even though she was not present when the threat was made.', \\\"recover, because she would not have suffered the emotional trauma had it not been for the neighbor's threat to her child's life.\\\", 'not recover, because she was not present when the threat was made.', 'not recover, because the neighbor did not touch her child.']\",\"question\":\"A child was playing with a ball in the front yard of his family's home. He accidentally threw the ball too far, and it hit a neighbor's new car. The neighbor was enraged and threatened to kill the child. The child ran inside his house and told his mother what had occurred. Although the child was only mildly upset by the incident, the child's mother was extremely distressed. She suffered an emotional breakdown and needed months of therapy to recover. If the mother sues the neighbor for the mental anguish suffered, she will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}