{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2576543-e518-5415-9d59-fea755318096","task_key":"default--test--10053","task_revision_id":"1","upstream_id":"10053","short_description":"A block is dragged along a table and experiences a frictional force, f, that…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['parallel to the table', 'equal to the frictional force', 'in the opposite direction of movement', 'perpendicular to the table', 'in the direction of movement', 'zero', 'equal to the gravitational force', 'neither parallel nor perpendicular to the table', 'always greater than the frictional force', 'always less than the frictional force']\",\"question\":\"A block is dragged along a table and experiences a frictional force, f, that opposes its movement. The force exerted on the block by the table is\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}