{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"160477d9-a204-55e3-aec2-d0af8d7533b7","task_key":"default--test--8469","task_revision_id":"1","upstream_id":"8469","short_description":"Which situation could be represented by the expression 6 x 2?","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Rocco read six books in two months.', 'Rocco hiked six miles each day for two days.', 'Rocco spent six hours on homework for two days.', 'Rocco has six siblings and two of them are girls.', 'Rocco had a total of six tennis balls in two cans.', 'Rocco had six shirts and washed two of them.', 'Rocco biked six miles and then continued for two more miles.', 'Rocco had six baseballs and gave away two of them.', 'Rocco had six apples and ate two of them.']\",\"question\":\"Which situation could be represented by the expression 6 x 2?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}