{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51727f7d-dbe9-509c-ac5c-e0eb65e960c8","task_key":"default--test--10293","task_revision_id":"1","upstream_id":"10293","short_description":"What single lens is equivalent to a thin converging lens of focal length 6 cm in…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['5 cm diverging lens', '15 cm diverging lens', '4 cm converging lens', '24 cm converging lens', '6 cm diverging lens', '20 cm diverging lens', '3 cm diverging lens', '10 cm converging lens', '15 cm converging lens', '8 cm converging lens']\",\"question\":\"What single lens is equivalent to a thin converging lens of focal length 6 cm in contact with a thin diverging lens of 10 cm focal length?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}