{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5ef0382-dac5-5579-8a52-caba7ff4b337","task_key":"all--test--d5ef0382-dac5-5579-8a52-caba7ff4b337","task_revision_id":"1","upstream_id":"","short_description":"A local politician starts a task force to reduce prejudice and hate crimes, and…","config":"all","split":"test","body":"{\"choices\":\"['I, II, and III', 'II, III, and IV', 'I, III, and IV', 'I, II, and IV']\",\"question\":\"A local politician starts a task force to reduce prejudice and hate crimes, and the task force provides recommendations for increased access to education, paid community improvement projects open to people from different, possibly conflicted, groups, and stricter anti-hate ordinances to be passed at the local level. These recommendations are based on:\\nI. Self-esteem hypothesis\\nII. Contact hypothesis\\nIII. Hypothesis\\nIV. Legal hypothesis\",\"subject\":\"college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}