{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64b699ca-637d-511b-9615-ed1f9f70caa4","task_key":"default--test--3965","task_revision_id":"1","upstream_id":"3965","short_description":"The air in a tank has a pressure of 640 mm of Hg at 23°C. When placed in…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['694 mm of Hg', '720 mm of Hg', '712 mm of Hg', '680 mm of Hg', '760 mm of Hg', '665 mm of Hg', '650 mm of Hg', '700 mm of Hg', '685 mm of Hg', '675 mm of Hg']\",\"question\":\"The air in a tank has a pressure of 640 mm of Hg at 23°C. When placed in sunlight the temperature rose to 48°C. What was the pressure in the tank?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}