{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e647ec98-59ed-5747-bd6b-45baafcd7293","task_key":"default--test--8236","task_revision_id":"1","upstream_id":"8236","short_description":"Jimmy and Kima are going on a trip. They will drive for three days. The first…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['180 + 440 + 220', '200 + 450 + 250', '150 + 450 + 250', '200 + 500 + 200', '150 + 400 + 250', '190 + 430 + 210', '200 + 400 + 200', '150 + 400 + 200', '200 + 400 + 250', '200 + 450 + 200']\",\"question\":\"Jimmy and Kima are going on a trip. They will drive for three days. The first day they will drive 182 miles. The second day they will drive 439 miles. The third day they will drive 217 miles. Which expression is the closest estimate of how many miles Jimmy and Kima will drive on their trip?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}