{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7d49b6e-484f-53b1-8f05-b8fcb837ee9b","task_key":"default--test--10034","task_revision_id":"1","upstream_id":"10034","short_description":"Bubbles of air escaping from a cylinder of compressed air at the bottom of a…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['3/1', '6/1', '1/3', '1/2', '2/1', '5/1', '4/1', '2/3', '5/2', '3/2']\",\"question\":\"Bubbles of air escaping from a cylinder of compressed air at the bottom of a pond 34 ft deep are observed to expand as they rise to the surface. Approximately how much do they increase in volume if it can be assumed that there is no change in temperature?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}