{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d523c1dd-163b-575f-9383-7a523060c0e4","task_key":"default--test--3268","task_revision_id":"1","upstream_id":"3268","short_description":"Which of the following is an example of a cline?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Males of some species have long antlers to fight other males for females.', 'A population of moths in a forest has evolved different color patterns depending on the type of tree they inhabit.', 'A species of fish has developed a unique color pattern in response to changes in water temperature.', 'Certain species of spiders have evolved different web-building behaviors depending on the availability of prey.', 'The hybrid tomato plant is stronger and produces better fruit than the pure genotype.', 'A certain type of beetle exhibits a wide range of body size, with larger individuals found in colder regions.', 'There are two distinct varieties in one population of snail that inhabits an island in the Pacific Ocean.', 'In one species of rabbit, the ones that evolved in the cold, snowy north are white, while the ones that evolved in the south are brown.', 'Some plants produce toxins that deter herbivores.', 'The frequency of a particular gene in a population of birds varies depending on the altitude of their habitat.']\",\"question\":\"Which of the following is an example of a cline?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}