{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6642a97b-8ec7-59dc-9d17-7e746ff66c52","task_key":"default--test--10135","task_revision_id":"1","upstream_id":"10135","short_description":"An automobile of mass 50 slugs accelerates from rest. During the first 10 sec,…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['120 ft/s, 1000 ft', '200 f', '130 ft/s, 900 ft', '180 f', '175 ft/s, 1150 ft', '145 ft/s, 1100 ft', '150 f', '190 ft/s, 950 ft', '160 ft/s, 1200 ft', '166 (2/3) f']\",\"question\":\"An automobile of mass 50 slugs accelerates from rest. During the first 10 sec, the resultant force acting on it is given by \\\\sumF = F_0 -kt, where F_0 = 200 lb, k = 10 lb/sec, and t is the time in seconds after the start. Find the velocity at the end of 10 sec, and the distance covered in this time.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}