{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"264164f8-a0b6-54f7-8712-8c82cc0ca0d2","task_key":"default--test--9919","task_revision_id":"1","upstream_id":"9919","short_description":"Two motorcycles are at rest and separated by 24.5 ft. They start at the same…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['(a) 8 sec, (b) 75 ft, (c) 22 ft/sec and 15 ft/sec', '(a) 7 sec, (b) 75 ft, (c) 21 ft/sec and 15 ft/sec', '(a) 8 sec, (b) 80 ft, (c) 24 ft/sec and 16 ft/sec', '(a) 9 sec, (b) 81 ft, (c) 23 ft/sec and 18 ft/sec', '(a) 7 sec, (b) 73.5 ft, (c) 21 ft/sec and 14 ft/sec', '(a) 6.5 sec, (b) 71 ft, (c) 19.5 ft/sec and 13 ft/sec', '(a) 6 sec, (b) 70 ft, (c) 20 ft/sec and 12 ft/sec', '(a) 10 sec, (b) 90 ft, (c) 25 ft/sec and 20 ft/sec', '(a) 5 sec, (b) 60 ft, (c) 18 ft/sec and 10 ft/sec', '(a) 7.5 sec, (b) 78 ft, (c) 22.5 ft/sec and 15 ft/sec']\",\"question\":\"Two motorcycles are at rest and separated by 24.5 ft. They start at the same time in the same direction, the one in the back having an acceleration of 3 ft/sec^2, the one in the front going slower at an acceleration of 2 ft/sec^2. (a) How long does it take for the faster cycle to overtake the slower, (b)How far does the faster machine go before it catches up? (c) How fast is each cycle going at this time?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}