{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"599381c9-f4ae-558e-91a3-a67488f4c626","task_key":"default--test--6583","task_revision_id":"1","upstream_id":"6583","short_description":"Which of the following statements is not true?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Humans cannot convert any fatty acids into docosahexaenoic acid', 'Flaxseeds are a rich source of docosahexaenoic acid', 'Docosahexaenoic acid levels are consistently higher in vegans than in omnivores', 'Vegan diets provide an abundance of docosahexaenoic acid naturally', 'Lack of docosahexaenoic acid in vegans causes visual and cognitive impairment', 'Docosahexaenoic acid is only found in animal-based foods', 'Docosahexaenoic acid can be synthesised from linolenic acid in vegans', 'All vegans require docosahexaenoic acid supplements to remain healthy', 'Microalgae can be used to synthesise docosahexaenoic acid', 'Docosahexaenoic acid is absent from vegan diets']\",\"question\":\"Which of the following statements is not true?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}