{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3af3ca72-57e3-5180-a6f7-63e0f5a16883","task_key":"default--test--9344","task_revision_id":"1","upstream_id":"9344","short_description":"A spherical light bulb 2 inches in diameter is imaged by a steel ball bearing…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.1 inches in diameter, located at a distance of 0.49 in. beneath the surface of the ball bearing', '0.1 inches in diameter, located at a distance of 10 inches away from the ball bearing', '0.05 inches in diameter, located at a distance of 2 in. above the surface of the ball bearing', '2 inches in diameter, located at the same position as the light bulb', '0.05 inches in diameter, located at a distance of 1 in. beneath the surface of the ball bearing', '0.1 inches in diameter, located at a distance of 1 in. above the surface of the ball bearing', '1 inch in diameter, located at a distance of 20 inches away from the ball bearing', '0.05 inches in diameter, located at a distance of 0.49 in. above the surface of the ball bearing', '0.05 inches in diameter, located at a distance of 0.49 in. beneath the surface of the ball bearing', '0.025 inches in diameter, located at a distance of 0.25 in. beneath the surface of the ball bearing']\",\"question\":\"A spherical light bulb 2 inches in diameter is imaged by a steel ball bearing located 20 inches away from the bulb. If the ball bearing is 1inch in diameter, where is the image of the light bulb and what is its diameter?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}