{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23761859-1c56-545f-baa3-295732fa6c4a","task_key":"default--test--9066","task_revision_id":"1","upstream_id":"9066","short_description":"A newly discovered planet has twice the density of the earth, but the…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['2.84 × 10^6 meters', '3.19 × 10^6 meters', '5.12 × 10^6 meters', '9.00 × 10^6 meters', '6.38 × 10^6 meters', '8.34 × 10^6 meters', '2.00 × 10^6 meters', '7.56 × 10^6 meters', '4.77 × 10^6 meters', '1.59 × 10^6 meters']\",\"question\":\"A newly discovered planet has twice the density of the earth, but the acceleration due to gravity on its surface is exactly the same as on the surface of the earth. What is its radius?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}