{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d149f006-03b4-543b-983b-f66ac01bc1cb","task_key":"default--test--10169","task_revision_id":"1","upstream_id":"10169","short_description":"The equation E = mc^2 indicates that energy","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['and mass are closely related.', 'is equal to the square of the speed of light.', 'equals moving mass.', 'is fundamentally different than mass.', 'equals mass divided by the speed of light squared.', 'and mass are not related at all.', 'equals the sum of mass and the speed of light squared.', 'is inversely proportional to mass.', 'equals mass moving at the speed of light squared.', 'equals mass multiplied by the speed of light.']\",\"question\":\"The equation E = mc^2 indicates that energy\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}