{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ba3c16a-7110-541d-8068-a7b579b9dad6","task_key":"default--test--10123","task_revision_id":"1","upstream_id":"10123","short_description":"Consider the earth to be a uniform solid sphere. If all of themass of the earth…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['60 hours, 2/3 of the initial kinetic energy is lost', '42 hours, 1/5 of the initial kinetic energy is lost', '54 hours, 4/5 of the initial kinetic energy is lost', '50 hours, 3/5 of the initial kinetic energy is lost', '36 hours, 1/4 of the initial kinetic energy is lost', '30 hours, 1/2 of the initial kinetic energy is lost', '28 hours, 1/6 of the initial kinetic energy is lost', '40 hours, 2/5 of the initial kinetic energy is lost', '24 hours, 1/3 of the initial kinetic energy is lost', '48 hours, 3/4 of the initial kinetic energy is lost']\",\"question\":\"Consider the earth to be a uniform solid sphere. If all of themass of the earth were suddenly movedradiallyinto athin-walled hollow spherical shell of the same radius, howlong would it take the new earth to rotate on its axis? What fraction of the initial kinetic energy of rotation wouldbe lostin this process?\",\"src\":\"stemez-Mechanics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}