{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb54d8bf-3734-5400-a414-3026f70be4e4","task_key":"all--test--fb54d8bf-3734-5400-a414-3026f70be4e4","task_revision_id":"1","upstream_id":"","short_description":"A man hosted several friends at his home for dinner. Two of the guests got into…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the host had the privilege of \\\"defense of others.\\\"', \\\"No, because the host's use of force was excessive under the circumstances.\\\", 'Yes, because the host has the privilege of host immunity.', 'No, because the host escalated the conflict by getting involved.']\",\"question\":\"A man hosted several friends at his home for dinner. Two of the guests got into an argument, and one of them got up from the table and lunged at the other with a large hunting knife. The host stepped in with a baseball bat and hit the attacker in the chest with it, preventing the imminent contact by the attacker with his target. The blow knocked the attacker down and caused several broken ribs. The injured guest sued the host for aggravated assault and battery. Is the court likely to grant the host's motion to dismiss the claim for failure to state a sufficient claim for relief?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}