{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68a03fe7-9833-5b94-b626-12f5666832d4","task_key":"all--test--68a03fe7-9833-5b94-b626-12f5666832d4","task_revision_id":"1","upstream_id":"","short_description":"Two scientists at a conference on evolution take to the stage on day 3 to argue…","config":"all","split":"test","body":"{\"choices\":\"['A study that shows that bodybuilders who train more have larger children.', 'A taxonomy study that shows long periods of stagnant growth followed by short burst of massive evolution.', 'A study that showed a species who were more successful due to the things they learned over their lifetime that they passed on to their children.', 'A study that showed a consistent amount of time between the emergence of each new species.']\",\"question\":\"Two scientists at a conference on evolution take to the stage on day 3 to argue their theories against one other. Each is a devout student of their own philosophy. The first scientist contends that organisms evolved via the increase of organs that were used the most during their time. They would then pass these on to subsequent generations. The second scientist, however, believed that advantages each organism possessed were absent for a long time, randomly occurred, and when they were beneficial, that organism would rapidly populate the population over a short period of time, evolutionarily speaking. Which of the following statements would strengthen the second scientist’s argument?\",\"subject\":\"college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}