{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2edc3451-8264-56c3-b485-29be4a29bf1d","task_key":"default--test--5094","task_revision_id":"1","upstream_id":"5094","short_description":"The nineteenth-century photographic process used to create a daguerreotype was…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['quick exposure time', 'ability to capture in low light', 'ability to capture sharp detail', 'long exposure time', 'use of glass negatives', 'use of liquid emulsion', 'capacity to be reproduced multiple times', 'use of paper negatives', 'use of color pigments', 'capacity for instant development']\",\"question\":\"The nineteenth-century photographic process used to create a daguerreotype was notable for its\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}