{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"92043d7f-1c87-570b-b454-b7213fe27a2f","task_key":"default--test--10160","task_revision_id":"1","upstream_id":"10160","short_description":"An elevator is rising. In order to determine the tension in the cables pulling…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Mass, velocity, and acceleration', 'Mass, velocity, height, and acceleration', 'Mass and velocity', 'Velocity, height, and acceleration', 'Velocity and acceleration', 'Mass, height, and acceleration', 'Mass and acceleration', 'Mass, height, and velocity', 'Mass, acceleration, and friction', 'Mass, velocity, acceleration, and friction']\",\"question\":\"An elevator is rising. In order to determine the tension in the cables pulling it, which values (for the elevator) would you need to know?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}