{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ccedfc0-cc00-5bdc-8106-a43e0a0780dc","task_key":"default--test--9672","task_revision_id":"1","upstream_id":"9672","short_description":"A gun is fired at a moving target. The bullet has a pro-jectile speed of 1200…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['25 degrees', '32 degrees', '15 degrees', '34.2 degrees', '21.3 degrees', '20 degrees', '27.5 degrees', '18.7 degrees', '23.4 degrees', '30 degrees']\",\"question\":\"A gun is fired at a moving target. The bullet has a pro-jectile speed of 1200 ft/sec. Both gun and target are on level ground. The target is moving 60 mph away from the gun and it is 30,000 ft from the gun at the moment of firing. Determine the angle of elevation \\\\texttheta needed for a direct hit.\",\"src\":\"stemez-Mechanics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}