{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44cbfb36-30d0-52a8-8759-7f23474be1d3","task_key":"default--test--2962","task_revision_id":"1","upstream_id":"2962","short_description":"The kangaroo rat never drinks water, yet it survives in some of the hottest and…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The kangaroo rat has a symbiotic relationship with a plant that provides it with water', 'The kangaroo rat consumes small pebbles that help it extract moisture from the food it eats', 'The kangaroo rat relies on a slow metabolism to minimize water loss', 'The kangaroo rat extracts water from the insects it eats', 'The kangaroo rat obtains all of its water from carbohydrate metabolism and its kidneys are modified to conserve as much water as possible.', 'The kangaroo rat drinks dew drops from plants', 'The kangaroo rat collects water in its fur during the night and licks it off', 'The kangaroo rat burrows deep underground to access moisture in the soil', 'The kangaroo rat has a special organ to absorb water from the air', 'The kangaroo rat stores water in its tail']\",\"question\":\"The kangaroo rat never drinks water, yet it survives in some of the hottest and driest regions of the world. Explain how it obtains and conserves water.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}