{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39560a2e-35ad-5bb2-be30-6e16cd31f161","task_key":"default--test--8115","task_revision_id":"1","upstream_id":"8115","short_description":"Mr. Bee's statistics class had a standard deviation of 11.2 on a standardized…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"[\\\"Mr. Bee's class performed twice as well as Mr. Em's.\\\", \\\"Mr. Em's class is less heterogeneous than Mr. Bee's.\\\", \\\"Mr. Em's class is more homogeneous than Mr. Bee's.\\\", \\\"Mr. Em's class did not do as well as Mr. Bee's.\\\", \\\"The standard deviation indicates that Mr. Bee's class had a lower test average.\\\", \\\"Mr. Bee's class is more homogeneous than Mr. Em's.\\\", \\\"The standard deviation indicates that Mr. Em's class had a higher test average.\\\", \\\"Mr. Bee's class did not do as well as Mr. Em's.\\\", \\\"Mr. Bee's class and Mr. Em's class performed equally well on the test.\\\"]\",\"question\":\"Mr. Bee's statistics class had a standard deviation of 11.2 on a standardized test, while Mr. Em's class had a standard deviation of 5.6 on the same test. Which of the following is the most reasonable conclusion concerning the two classes' performance on the test?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}