{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"456b3055-bf52-5819-a058-7f6ab8cee480","task_key":"all--test--456b3055-bf52-5819-a058-7f6ab8cee480","task_revision_id":"1","upstream_id":"","short_description":"In a certain country, a person must be at least 16 years old to drive a car and…","config":"all","split":"test","body":"{\"choices\":\"['II only', 'I and II only', 'I and III only', 'II and III only']\",\"question\":\"In a certain country, a person must be at least 16 years old to drive a car and must be at least 18 years old to vote. The variable age represents the age of a person as an integer. Which of the following expressions evaluates to true if the person is old enough to drive but not old enough to vote, and evaluates to false otherwise?\\n I. (age ≥ 16) AND (age ≤ 18)\\n II. (age ≥ 16) AND (NOT(age ≥ 18))\\n III. (age < 18) AND (NOT(age < 16))\",\"subject\":\"high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}