{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b692616-47b5-5ebc-a5b8-c98db49bcad7","task_key":"default--test--9408","task_revision_id":"1","upstream_id":"9408","short_description":"A body is whirled in a vertical circle by means of a cord attached at the center…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['9.8 ft/s', '10.5 ft/s', '12.3 ft/s', '11.0 ft/s', '6.7 ft/s', '7.2 ft/s', '5.4 ft/s', '14.8 ft/s', '4.9 ft/s', '8.6 ft/s']\",\"question\":\"A body is whirled in a vertical circle by means of a cord attached at the center of the circle. At what minimum speed must it travel at the top of the path in order to keep the cord barely taut, i.e., just on the verge of collapse? Assume radius of circle to be 3 ft.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}