{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e877e20-c5c1-5f21-bec4-be5ff756f23b","task_key":"all--test--5e877e20-c5c1-5f21-bec4-be5ff756f23b","task_revision_id":"1","upstream_id":"","short_description":"The colonic microflora can be considered as \"an organ within an organ\". Which of…","config":"all","split":"test","body":"{\"choices\":\"['Measurement of daily dietary fibre intake underestimates the amount of fermentable material entering the colon each day', 'All of the options given are correct', 'Acetate, propionate and butyrate are the chief organic acids produced by anaerobic colic fermentation', 'The establishment of an \\\"acetate buffer\\\" in the colonic lumen suppresses the growth of some pathogenic bacteria.']\",\"question\":\"The colonic microflora can be considered as \\\"an organ within an organ\\\". Which of the following statements best describes the functions of the microbiota?\\n\",\"subject\":\"nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}