{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6f40a0a-c968-57ed-a73a-90de39d31998","task_key":"default--test--9304","task_revision_id":"1","upstream_id":"9304","short_description":"Spring scales are used to measure the net force applied to an object; a sonic…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Inertial mass', 'Gravitational mass', 'Kinetic energy', 'Velocity', 'Weight', 'Friction', 'Force', 'Momentum', 'Distance', 'Speed']\",\"question\":\"Spring scales are used to measure the net force applied to an object; a sonic motion detector is used to measure the object’s resulting acceleration. A graph is constructed with the net force on the vertical axis and the acceleration on the horizontal axis. Which of the following quantities is directly measured using the slope of this graph?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}