{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3bf7c56a-0800-53ca-ba2c-7cf3f94e1257","task_key":"default--test--3075","task_revision_id":"1","upstream_id":"3075","short_description":"Why are the stem reptiles especially important in evolution?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They exclusively evolved into modern reptiles without any other descendants', 'They were the first reptiles to return to an aquatic lifestyle', 'They gave rise to only the dinosaurs', 'They were the only reptiles to develop flight', 'They existed only in the Mesozoic Era', 'They were the last reptiles to evolve', 'They represent the peak of reptilian evolution in terms of size and complexity', 'They were exclusively herbivorous, unlike other reptiles', 'They were the first reptiles to develop endothermy (warm-bloodedness)', 'They were the ancestral form of a variety of important animals, including the mammals.']\",\"question\":\"Why are the stem reptiles especially important in evolution?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}