{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c58df611-901e-591d-b807-28db080c4c4d","task_key":"default--test--10189","task_revision_id":"1","upstream_id":"10189","short_description":"A solution of camphor in alcohol in a tube 20 cm long rotates the plane of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['.450 grams per cc', '.615 grams per cc', '.306 grams per cc', '.150 grams per cc', '.350 grams per cc', '.500 grams per cc', '.405 grams per cc', '.275 grams per cc', '.250 grams per cc', '.750 grams per cc']\",\"question\":\"A solution of camphor in alcohol in a tube 20 cm long rotates the plane of vibration of light passing through it 33°. What must the density of camphor be in the solution? The specific rotation of camphor is + 54°.\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}