{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bea6747-000f-527c-bcf7-286fd10d9dd3","task_key":"default--test--3170","task_revision_id":"1","upstream_id":"3170","short_description":"Compare the methods of obtaining oxygen in the earthworm, the frog, the fish and…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Earthworms use gills, frogs use both gills and lungs, fish use lungs, and humans use skin.', 'Earthworms and humans use lungs, frogs and fish use gills.', 'All use gills for obtaining oxygen.', 'Earthworms and humans use gills, frogs use skin, and fish use lungs.', 'Earthworms use lungs, frogs use gills, fish use skin, and humans use gills.', 'Earthworms and fish use skin, frogs use lungs, and humans use gills.', 'Earthworms use skin and lungs, frogs use gills, fish use lungs, and humans use skin.', 'Earthworms respire through their skin, frogs use their skin and lungs, fish use gills, and humans use lungs.', 'Earthworms use gills, frogs use lungs, fish use skin, and humans use lungs.']\",\"question\":\"Compare the methods of obtaining oxygen in the earthworm, the frog, the fish and man.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}