{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29f53944-e100-5e77-9465-ef3102a37f2f","task_key":"default--test--10261","task_revision_id":"1","upstream_id":"10261","short_description":"A simple and correct way to comprehend satellites orbiting Earth is to view them…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['defying gravity through an internal mechanism.', 'in mechanical equilibrium with a net force of zero.', 'being propelled by constant rocket thrust.', 'beyond the main pull of Earth gravity.', 'moving in a straight line at a constant speed.', 'having sufficient tangential velocities to fall around rather than into Earth.', 'being pushed by solar winds.', 'suspended in the atmosphere by helium balloons.', 'balanced between gravitational and centripetal forces.', 'being held up by the magnetic field of the Earth.']\",\"question\":\"A simple and correct way to comprehend satellites orbiting Earth is to view them as\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}