{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88cbba53-85cc-50cb-ac8c-5e6dbeb91e60","task_key":"default--test--9574","task_revision_id":"1","upstream_id":"9574","short_description":"The lens of a camera has a focal length of 6 cm and the camera is focused for…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.5 centimeters away from the film', '1.5 centimeters away from the film', '0.7 centimeters away from the film', '0.3 centimeters away from the film', '0.4 centimeters towards the film', '0.2 centimeters away from the film', '0.8 centimeters towards the film', '0.1 centimeters towards the film', '0.6 centimeters towards the film', '1.0 centimeters away from the film']\",\"question\":\"The lens of a camera has a focal length of 6 cm and the camera is focused for very distant objects. Through what distance must the lens be moved, and in what direction, when the focus is readjusted for an object 2 m from the lens?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}