{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f455aaab-0fa1-5190-9218-40a36b654c94","task_key":"all--test--f455aaab-0fa1-5190-9218-40a36b654c94","task_revision_id":"1","upstream_id":"","short_description":"As a result of substance abuse throughout adolescence, a young adult suffers…","config":"all","split":"test","body":"{\"choices\":\"['I only', 'III only', 'I and III only', 'II and III only']\",\"question\":\"As a result of substance abuse throughout adolescence, a young adult suffers from a number of psychological symptoms reflecting diminished executive functioning. Which of the following are likely true of this patient?\\nI. Pathological changes to the prefrontal cortex.\\nII. Increased susceptibility to auditory hallucinations.\\nIII. Reduced behavioral impulse control.\",\"subject\":\"college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}