{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3a6a890-3979-50de-87bf-21fd89942b90","task_key":"default--test--8131","task_revision_id":"1","upstream_id":"8131","short_description":"Kim rode her bicycle 135 miles in 9 weeks, riding the same distance each week.…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Eric rode 7 more miles per week than Kim rode.', 'Kim rode 5 more miles per week than Eric rode.', 'Kim rode 3 more miles per week than Eric rode.', 'Eric and Kim rode the same number of miles per week.', 'Eric rode 17 more miles per week than Kim rode.', 'Eric rode 13 more miles per week than Kim rode.', 'Kim rode 9 more miles per week than Eric rode.', 'Kim rode 11 more miles per week than Eric rode.', 'Kim rode 15 more miles per week than Eric rode.', 'Eric rode 2 more miles per week than Kim rode.']\",\"question\":\"Kim rode her bicycle 135 miles in 9 weeks, riding the same distance each week. Eric rode his bicycle 102 miles in 6 weeks, riding the same distance each week. Which statement correctly compares the number of miles per week they rode?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}