{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70c68d03-dea2-54a6-8473-49a2e8323e95","task_key":"default--test--7804","task_revision_id":"1","upstream_id":"7804","short_description":"A manufacturer of motor oil is interested in testing the effects of a newly…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Independent samples t-test for population proportions', 'Two-way analysis of variance (ANOVA)', 'One-way analysis of variance (ANOVA)', 'Repeated measures analysis of variance (ANOVA)', 'Paired t-test for population proportions', 'Independent samples comparison of population proportions', 'Matched pairs comparison of population proportions', 'Chi-square test for independence', 'Matched pairs comparison of population means', 'Independent samples comparison of population means']\",\"question\":\"A manufacturer of motor oil is interested in testing the effects of a newly developed additive on the lifespan of an engine. Twenty-five different engine types are selected at random and each one is tested using oil with the additive and oil without the additive. What type of analysis will yield the most useful information?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}