{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43f529d5-1ee8-5cf2-900c-8a7d5d6c7fb8","task_key":"default--test--9737","task_revision_id":"1","upstream_id":"9737","short_description":"When a wave moves from shallow water to deep water, the","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['frequency does not change, the wavelength increases, and the speed increases', 'frequency decreases, the wavelength increases, and the speed does not change', 'frequency increases, the wavelength increases, and the speed decreases', 'frequency increases, the wavelength does not change, and the speed increases', 'frequency increases, the wavelength decreases, and the speed does not change', 'frequency does not change, the wavelength increases, and the speed decreases', 'frequency does not change, the wavelength decreases, and the speed decreases', 'frequency decreases, the wavelength increases, and the speed increases', 'frequency decreases, the wavelength decreases, and the speed increases', 'frequency does not change, the wavelength does not change, and the speed decreases']\",\"question\":\"When a wave moves from shallow water to deep water, the\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}