{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4b915a0-1a70-5414-9039-40deb89568cc","task_key":"default--test--9926","task_revision_id":"1","upstream_id":"9926","short_description":"A person stands on a scale in an elevator. He notices that the scale reading is…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['It is moving in a circular motion.', 'It is moving upward and speeding up.', 'It is moving upward at a constant speed.', 'It is moving downward at a constant speed.', 'It is stationary at the top floor.', 'It is stationary at the bottom floor.', 'It is moving upward and slowing down.', 'It is moving sideways.', 'It is moving downward and slowing down.']\",\"question\":\"A person stands on a scale in an elevator. He notices that the scale reading is less than his usual weight. Which of the following could possibly describe the motion of the elevator?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}