{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51bcc353-d06a-56be-8b1a-c0ca30d34d9c","task_key":"default--test--9423","task_revision_id":"1","upstream_id":"9423","short_description":"In a hydraulic press operation, the output piston cannot","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['function without fluid', 'exceed the input piston’s speed', 'produce decreased energy', 'exceed the pressure input', 'operate without the input piston', 'move at the same speed as the input piston', 'exceed the force input', 'move farther than the input piston', \\\"exceed the input piston's weight\\\", 'produce increased energy']\",\"question\":\"In a hydraulic press operation, the output piston cannot\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}