{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cffc2ca-1b3e-5046-a006-fb2503406546","task_key":"default--test--9340","task_revision_id":"1","upstream_id":"9340","short_description":"An elevator is descending with an acceleration of 5 ft\\bullets^-2. The shade on…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.5 s', '0.3 s', '0.7 s', '1 s', '0.6 s', '2 s', '1.2 s', '1.5 s', '0.82 s', '1.8 s']\",\"question\":\"An elevator is descending with an acceleration of 5 ft\\\\bullets^-2. The shade on the ceiling light falls to the floor of the elevator, 9 ft below. At the instant that it falls, one of the passengers sees it, and realizes that the shade will hit his foot. How long does he have to get his foot out of the way?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}