{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96304ce1-90a0-51e6-8d6c-d3023b877081","task_key":"default--test--3082","task_revision_id":"1","upstream_id":"3082","short_description":"What controls the process of molting incrustacea?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Water temperature alone without hormonal influence', 'The age of the crustacean', 'Environmental factors', 'Social interactions with other crustaceans', 'Diet of the crustacean', 'The salinity levels of the water', 'The lunar cycle affecting tidal patterns', 'A hormone released from nerve cells', 'The presence of predators in the environment', 'Physical growth of the crustacean']\",\"question\":\"What controls the process of molting incrustacea?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}