{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b163b69f-39cd-55be-a26b-0fc1120b1b0d","task_key":"default--test--9894","task_revision_id":"1","upstream_id":"9894","short_description":"A car covers a distance of 30 miles in 1/2 hour. What is its speed in miles per…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['30 mi per hr, 60 ft per sec', '90 mi per hr, 132 ft per sec', '75 mi per hr, 110 ft per sec', '60 mi per hr, 55 ft per sec', '45 mi per hr, 88 ft per sec', '45 mi per hr, 66 ft per sec', '120 mi per hr, 176 ft per sec', '30 mi per hr, 44 ft per sec', '60 mi per hr, 88 ft per sec', '60 mi per hr, 100 ft per sec']\",\"question\":\"A car covers a distance of 30 miles in 1/2 hour. What is its speed in miles per hour and in feet per second?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}