{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d1435554-15b8-5b25-9b8c-223ebd593877","task_key":"default--test--9117","task_revision_id":"1","upstream_id":"9117","short_description":"A heavy rock and a light rock in free fall (zero air resistance) have the same…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['air resistance is always zero in free fall.', 'volume of both rocks is the same.', 'force due to gravity is the same on each.', 'gravitational constant is the same for both rocks.', 'force due to gravity is zero in free fall.', 'ratio of force to mass is the same.', 'inertia of both rocks is the same.', 'weight of both rocks is the same.', 'density of both rocks is the same.', 'mass of both rocks is the same.']\",\"question\":\"A heavy rock and a light rock in free fall (zero air resistance) have the same acceleration. The heavy rock doesn't have a greater acceleration because the\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}