{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7de0b73c-b880-569b-906f-e75be24dd0d7","task_key":"default--test--3483","task_revision_id":"1","upstream_id":"3483","short_description":"How are whales adapted for their life in cold waters?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They produce heat through rapid movement and have a layer of reflective scales', 'They utilize a bio-antifreeze in their blood to prevent ice crystal formation', 'They have fins for swimming and excrete a special anti-freezing substance', 'They have gills and a streamlined body', 'They have a temperature regulating system and a tremendous amount of subcutaneous fat serving an insulating function.', 'They have specialized lungs that warm incoming cold water', 'They migrate to warmer waters during colder seasons to maintain body temperature', 'They have fur and hibernate in winter']\",\"question\":\"How are whales adapted for their life in cold waters?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}