{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f70e2bbb-5a0a-5a19-8641-f2d518eb9c69","task_key":"default--test--9142","task_revision_id":"1","upstream_id":"9142","short_description":"Shot towers were popular in the eighteenth and nineteenth centuries for dropping…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['1.89 $\\\\\\\\mathrm{~mm}$', '3.52 $\\\\\\\\mathrm{~mm}$', '5.14 mm directly south due to Coriolis effect', '2.68 $\\\\\\\\mathrm{~mm}$', \\\"3.02 mm to the North due to Earth's rotation\\\", \\\"1.15 mm to the West due to Earth's rotation\\\", '2.26 $\\\\\\\\mathrm{~mm}$', \\\"6.33 mm to the South-East due to combined effects of Earth's rotation and prevailing winds\\\", \\\"4.73 mm to the East due to Earth's rotation\\\", '0.76 mm to the East due to local wind conditions']\",\"question\":\"Shot towers were popular in the eighteenth and nineteenth centuries for dropping melted lead down tall towers to form spheres for bullets. The lead solidified while falling and often landed in water to cool the lead bullets. Many such shot towers were built in New York State. Assume a shot tower was constructed at latitude $42^{\\\\circ} \\\\mathrm{N}$, and the lead fell a distance of $27 \\\\mathrm{~m}$. In what direction and how far did the lead bullets land from the direct vertical?\",\"src\":\"scibench-class\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}