{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e403835b-1fe4-594a-85f5-9a5407cdffe3","task_key":"default--test--3340","task_revision_id":"1","upstream_id":"3340","short_description":"Vinegar is classified as an acid. What does this tell you about the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Vinegar has a lower concentration of hydrogen ions than hydroxide ions', 'Vinegar has an excess of hydrogen ions with respect to hydroxide ions', 'Vinegar has a higher concentration of hydroxide ions than water', 'Vinegar has no hydroxide ions', 'Vinegar does not contain hydrogen ions', 'Vinegar has an excess of hydroxide ions with respect to hydrogen ions', 'Vinegar has an equal concentration of hydrogen ions to water', 'Vinegar has a fluctuating concentration of hydrogen and hydroxide ions', \\\"Vinegar's hydrogen and hydroxide ion concentrations are not measurable\\\", 'Vinegar has an equal concentration of hydrogen and hydroxide ions']\",\"question\":\"Vinegar is classified as an acid. What does this tell you about the concentration of the hydrogen ions and the hydroxide ions?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}