{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a14f1d9-1c63-5813-be78-214f8103a33e","task_key":"default--test--9683","task_revision_id":"1","upstream_id":"9683","short_description":"At the present time, the temperature of the universe (i.e., the microwave…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['three times as distant as they are today', 'separated by double the distances as they are today', 'one-quarter as distant as they are today', 'separated by about the same distances as they are today', 'clustered more densely than they are today', 'not yet formed', 'ten times as distant as they are today', 'four times as distant as they are today', 'two times as distant as they are today', 'one-half as distant as they are today']\",\"question\":\"At the present time, the temperature of the universe (i.e., the microwave radiation background) is about 3 K. When the temperature was 12 K, typical objects in the universe, such as galaxies, were\",\"src\":\"ori_mmlu-college_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}