{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"678690fd-e8a2-5ada-a082-b36bcd5a38c8","task_key":"default--test--4734","task_revision_id":"1","upstream_id":"4734","short_description":"Genetic studies of the two primary varieties of rice, japonica (short-grained),…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['each is more closely related to one another than their local wild ancestors, suggesting that these modern varieties of rice were developed recently in east Asia.', 'each is more closely related to local wild ancestors than one another, suggesting that these modern varieties of rice were domesticated separately in east Asia and south Asia.', 'each is unrelated to one another, suggesting that these modern varieties of rice were domesticated independently in east Asia and western Europe.', 'each is closely related to their local wild ancestors, suggesting that domestication must have occurred very recently and no earlier than 2,000 years ago.']\",\"question\":\"Genetic studies of the two primary varieties of rice, japonica (short-grained), and indica (long-grained), suggest that:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}