{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9393719b-eaad-5f05-824a-841c20cd26c8","task_key":"default--test--9943","task_revision_id":"1","upstream_id":"9943","short_description":"The definition of magnetic inclination is the angle between","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['the earth’s magnetic field at any point and the plane of the equator', 'magnetic north and geographic north', 'magnetic north and geographic south', 'the earth’s magnetic field at any point and the plane of the ecliptic', 'the earth’s magnetic field at any point and the geographic south', 'magnetic south and geographic north', 'the earth’s magnetic field at any point and the geographic north', 'the earth’s magnetic field at any point and the vertical', 'the earth’s magnetic field at any point and the horizontal', 'magnetic south and geographic south']\",\"question\":\"The definition of magnetic inclination is the angle between\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}