{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e179068-bef2-53bf-b341-70bbfdef57c3","task_key":"default--test--4999","task_revision_id":"1","upstream_id":"4999","short_description":"Which of the following was the greatest advantage of using fire in human…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Fire can be used in warfare and to clear fields.', 'Fire was used to create the first forms of currency.', 'Fire allowed early humans to fly kites at night.', 'Fire was used to create intricate cave paintings.', 'Fire makes meat more digestible and kills bacteria.', 'Fire was used to scare away insects and pests.']\",\"question\":\"Which of the following was the greatest advantage of using fire in human evolution?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}