{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9139cb66-0d9a-57d9-bb8a-31f27031ffb6","task_key":"default--test--3149","task_revision_id":"1","upstream_id":"3149","short_description":"How are whales adapted for their life in cold waters?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Whales have a specialized gill system to regulate temperature.', 'Whales are cold-blooded and adjust their body temperature to the water temperature.', 'Whales have a thick fur coat to keep them warm.', 'Whales have a unique enzyme that prevents their body fluids from freezing.', 'Whales maintain a nearly constant internal body temperature and have a tremendous amount of subcutaneous fat serving an insulating function.', 'Whales possess an advanced circulatory system that directs warm blood to the skin surface.', 'Whales have expandable bladders that they fill with warm water to maintain body temperature.']\",\"question\":\"How are whales adapted for their life in cold waters?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}