{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4e04c46e-fc1d-57c1-81c2-8935777928d3","task_key":"all--test--4e04c46e-fc1d-57c1-81c2-8935777928d3","task_revision_id":"1","upstream_id":"","short_description":"Identify the conclusion of the following argument. From a historical point of…","config":"all","split":"test","body":"{\"choices\":\"['But even then, the importance of DNA was not really understood until the 1950s.', 'The importance of DNA was not really understood until the 1950s.', 'From a historical point of view, genetics is a young science.', 'The principles that govern inheritance traits were described less than 150 years ago.']\",\"question\":\"Identify the conclusion of the following argument. From a historical point of view, genetics is a young science. The principles that govern inheritance traits were described less than 150 years ago. Furthermore, around the turn of the twentieth century, the laws of inheritance were rediscovered. But even then, the importance of DNA was not really understood until the 1950s.\",\"subject\":\"formal_logic\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}