{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"015f2942-10c6-551d-beb9-ff783a912a37","task_key":"all--test--015f2942-10c6-551d-beb9-ff783a912a37","task_revision_id":"1","upstream_id":"","short_description":"Which of the following correctly categorizes the components of Ivan Pavlov's…","config":"all","split":"test","body":"{\"choices\":\"['The dogs (subjects) were presented with food (CS) while also being presented with the sights and sounds of the presenter (UCS). After a time, the dogs salivated in response to the presenter (UCR).', 'The dogs (subjects) were presented with food (UCR) while also being presented with the sights and sounds of the presenter (CR). After a time, the dogs salivated in response to the presenter (CS).', 'The dogs (subjects) were presented with food (UCS) while also being presented with the sights and sounds of the presenter (CS). After a time, the dogs salivated in response to the presenter (CR).', 'The dogs (subjects) were presented with food (CR) while also being presented with the sights and sounds of the presenter (UCR). After a time, the dogs salivated in response to the presenter (UCS).']\",\"question\":\"Which of the following correctly categorizes the components of Ivan Pavlov's research on conditioning?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}