{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f65d6c9-0e82-5cd5-a1b4-efae4c3f3902","task_key":"default--test--10259","task_revision_id":"1","upstream_id":"10259","short_description":"When you drop a ball from rest it accelerates downward at 9.8 m/s^2. If you…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['more than 9.8 m/s2', 'less than 9.8 m/s2', 'Greater if the ball is thrown with a force', '19.6 m/s2', 'Cannot say unless the speed of throw is given.', 'Dependent on the angle of the throw', 'Not accelerated, it maintains the speed of the throw.', 'Zero, as the ball is in free fall', 'Equal to the speed of the throw', '9.8 m/s2']\",\"question\":\"When you drop a ball from rest it accelerates downward at 9.8 m/s^2. If you instead throw it downward assuming no air resistance its acceleration immediately after leaving your hand is\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}