{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69353327-8de2-5a42-8df6-153a3e707091","task_key":"default--test--3422","task_revision_id":"1","upstream_id":"3422","short_description":"If a living frog muscle is dissected out from a frog's body and placed in a…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The muscle cells contract due to the presence of carbon dioxide.', 'The muscle contracts because of the decrease in temperature when the container is opened.', 'The muscle cells are reanimated by the light entering the container when it is opened.', 'The muscle cells contract due to the release of lactic acid in the absence of oxygen.', 'The muscle cells require the presence of an inert gas, such as argon, to contract.', 'The muscle cells need nitrogen to contract.', 'The muscle cells contract due to the introduction of water.', 'The presence of oxygen enables the muscle cells to undergo aerobic respiration and produce more ATP, regaining its ability to contract.', 'Muscle contraction is restored due to increased humidity within the container.', 'The muscle cells contract because of the introduction of a new energy source when the container is opened.']\",\"question\":\"If a living frog muscle is dissected out from a frog's body and placed in a closed container without oxygen, it can contract for a time when stimulated. Then it stops. But if the container is opened, it soon regains the ability to contract. Comment.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}