{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87055eb5-b0de-5f1d-a95a-39b1cc21c1db","task_key":"default--test--9551","task_revision_id":"1","upstream_id":"9551","short_description":"Satellites in circular Earth orbit maintain a constant speed because…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['changing with the altitude of the satellite', 'without a component of force along their circular path', 'practically zero', 'appreciably weaker than at Earth’s surface', 'equal to the centrifugal force acting on them', \\\"stronger than at Earth's surface\\\", 'dependent on the speed of the satellite', 'always directed towards the center of the Earth', 'zero', 'dependent on the mass of the satellite']\",\"question\":\"Satellites in circular Earth orbit maintain a constant speed because gravitational force on them is\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}