{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76c63ee0-c4e8-5a4f-b71f-840ea026286e","task_key":"default--test--4822","task_revision_id":"1","upstream_id":"4822","short_description":"Virtually all of the crops important in the European Neolithic:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['were primarily animal products, not plant-based.', 'were imported from other regions.', 'were domesticated before groups became sedentary.', 'were domesticated at the same time at communal farms.', 'were mainly wild crops that were not domesticated until the Bronze Age.', 'were independently domesticated in Europe', 'were domesticated only after the advent of sedentary lifestyles.', 'were all domesticated in Europe by nomadic tribes.', 'were all domesticated in Asia and then brought to Europe.']\",\"question\":\"Virtually all of the crops important in the European Neolithic:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}