{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"258314bd-2954-5f1b-b225-65cbc53bda1a","task_key":"all--test--258314bd-2954-5f1b-b225-65cbc53bda1a","task_revision_id":"1","upstream_id":"","short_description":"A man invited several friends to come over to his house to watch a movie on…","config":"all","split":"test","body":"{\"choices\":\"['No duty of care.', 'A duty to inspect the premises for unknown dangers and disclose their existence to others.', 'A duty to warn of any known dangerous condition on the premises.', 'An absolute duty of care.']\",\"question\":\"A man invited several friends to come over to his house to watch a movie on television. While they were watching the movie, freezing rain fell outside, coating everything with a thin layer of very slippery ice. When the movie ended, one of the man's guests slipped on the ice as soon as he took a step out of the door and onto the front stoop of the man's house. Which of the following would best describe the duty of care owed by the man to his guest?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}