{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"155c6d06-3a06-58c7-b23e-02fce9fd8848","task_key":"default--test--9610","task_revision_id":"1","upstream_id":"9610","short_description":"A curling stone of mass 1 slug is sent off along the ice and comes to rest after…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Initial velocity: 10 ft/s, Time of motion: 20 s', 'Initial velocity: 18 ft/s, Time of motion: 36 s', 'Initial velocity: 8 ft/s, Time of motion: 25 s', 'Initial velocity: 22 ft/s, Time of motion: 44 s', 'Initial velocity: 30 ft/s, Time of motion: 60 s', 'Initial velocity: 15 ft/s, Time of motion: 30 s', 'Initial velocity: 12 ft/s, Time of motion: 24 s', 'Initial velocity: 20 ft/s, Time of motion: 10 s', 'Initial velocity: 5 ft/s, Time of motion: 40 s', 'Initial velocity: 25 ft/s, Time of motion: 50 s']\",\"question\":\"A curling stone of mass 1 slug is sent off along the ice and comes to rest after 100 ft. If the force of kinetic friction between stone and ice is 1/2 lb, with what velocity did the stone start, and how long did it take to come to rest?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}