{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e4a3554-826a-59e0-aee8-1f2b27d2719b","task_key":"default--test--9815","task_revision_id":"1","upstream_id":"9815","short_description":"A man stands on a platform scale in an elevator. The elevator moves upward,…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The force of the scale on the man', 'The force of the Earth on the elevator', 'The force of the man on the scale', 'The force of the Earth on the man', 'The force of the elevator cable on the elevator', 'The force of gravity on the man', 'The force of the elevator on the man', 'The force of the man on the Earth', 'The force of the man on the elevator cable', 'The force of the elevator cable on the man']\",\"question\":\"A man stands on a platform scale in an elevator. The elevator moves upward, speeding up. What is the action-reaction force pair to the man’s weight?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}