{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a9d695d-44e6-51e8-9eec-d5d877a36e31","task_key":"default--test--3519","task_revision_id":"1","upstream_id":"3519","short_description":"During the courtship of the common tern, the male presents afish to another…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Courtship is primarily for entertainment purposes among terns', 'Courtship is not necessary for reproduction', 'Courtship is a way to show dominance', 'Courtship is a random behavior with no impact on reproduction', 'Courtship is necessary for physical transformation required for reproduction', 'Courtship is only for attracting the opposite sex', 'Courtship serves for sexual identification, decreases aggressive tendencies for mating, necessary for some animals to produce eggs and establish species identification.', 'Courtship is used to mark territory between competing terns', 'Courtship is solely to determine the healthiest mate', 'Courtship is a way for terns to practice flying skills']\",\"question\":\"During the courtship of the common tern, the male presents afish to another tern. Account for this behavior and discuss whycourtship in general may benecesaryfor reproduction.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}