{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80635e2c-dd52-5483-8097-60a7806ee96c","task_key":"default--test--10260","task_revision_id":"1","upstream_id":"10260","short_description":"The rotational inertia of an object is greater when most of the mass is located","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['at the ends of the object', 'on the rotational axis', 'at the bottom of the object', 'in the center of the object', 'at the top of the object', 'near the rotational axis', 'equally distributed throughout the object', 'away from the axis', 'at one side of the object', 'off center']\",\"question\":\"The rotational inertia of an object is greater when most of the mass is located\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}