{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e86b2686-c83d-5c26-97bb-5ec16ab580c6","task_key":"default--test--3516","task_revision_id":"1","upstream_id":"3516","short_description":"How does a facultative anaerobe differ from anobligatory anaerobe?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Facultative anaerobes can survive in oxygen-rich environments by entering a dormant state', 'Facultative anaerobes can live either in the presence or absence of oxygen', 'Facultative anaerobes can only live in the presence of oxygen', 'Facultative anaerobes die in the presence of oxygen', 'Obligatory anaerobes can tolerate low levels of oxygen for short periods', 'Obligatory anaerobes can live either in the presence or absence of oxygen', 'Obligatory anaerobes use oxygen for respiration when it is available', 'Facultative anaerobes cannot live without oxygen', 'Facultative anaerobes can live without oxygen but prefer environments with low oxygen levels']\",\"question\":\"How does a facultative anaerobe differ from anobligatory anaerobe?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}