{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f798f21-b8de-55fe-814a-2617e4daeda8","task_key":"default--test--7903","task_revision_id":"1","upstream_id":"7903","short_description":"A manufacturer of ready-bake cake mixes is interested in designing an experiment…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['A randomized block design, blocked on temperature and type of pan, with 12 treatment groups', 'A completely randomized design with 6 treatment groups', 'A randomized block design, blocked on type of oven, with 24 treatment groups', 'A randomized block design, blocked on temperature, with six treatment groups', 'A completely randomized design with 18 treatment groups', 'A randomized block design, blocked on type of oven, with 12 treatment groups', 'A completely randomized design with 24 treatment groups', 'A randomized block design, blocked on type of pan, with 12 treatment groups', 'A completely randomized design with 12 treatment groups', 'A completely randomized design with nine treatment groups']\",\"question\":\"A manufacturer of ready-bake cake mixes is interested in designing an experiment to test the effects of four different temperature levels (300, 325, 350, and 375F), two different types of pans (glass and metal), and three different types of ovens (gas, electric, and microwave) on the texture of its cakes, in all combinations. Which of the following below is the best description of the design of the necessary experiment?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}