{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6b4d558-f46a-56b7-87d4-758b6bb6851b","task_key":"default--test--9983","task_revision_id":"1","upstream_id":"9983","short_description":"A relay is to be controlled by a vacuum photocell, actuated by the light passing…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['78.7 cm', '28.7 cm', '68.7 cm', '88.7 cm', '38.7 cm', '48.7 cm', '18.7 cm', '58.7 cm', '8.7 cm', '98.7 cm']\",\"question\":\"A relay is to be controlled by a vacuum photocell, actuated by the light passing through an aperture measuring 15 mm by 40 mm. At least 0.2 lumen must strike the photocell to operate the relay. What is the maximum permissible distance from the aperture to a uniform point source of intensity 50 candles, if the light from the source is to trigger the relay?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}