{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c20b5e6-f25a-52b9-962a-7a24d1663985","task_key":"all--test--9c20b5e6-f25a-52b9-962a-7a24d1663985","task_revision_id":"1","upstream_id":"","short_description":"A manufacturer of ready-bake cake mixes is interested in designing an experiment…","config":"all","split":"test","body":"{\"choices\":\"['A completely randomized design with nine treatment groups', 'A completely randomized design with 24 treatment groups', 'A randomized block design, blocked on temperature, with six treatment groups', 'A randomized block design, blocked on type of pan, with 12 treatment groups']\",\"question\":\"A manufacturer of ready-bake cake mixes is interested in designing an experiment to test the effects of four different temperature levels (300, 325, 350, and 375F), two different types of pans (glass and metal), and three different types of ovens (gas, electric, and microwave) on the texture of its cakes, in all combinations. Which of the following below is the best description of the design of the necessary experiment?\",\"subject\":\"high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}