{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3fa515ad-73a2-5618-82aa-a5c78ffcfa84","task_key":"default--test--6303","task_revision_id":"1","upstream_id":"6303","short_description":"Laryngeal oedema is usually confined to the larynx above the vocal folds because","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['laryngeal oedema is not confined to any particular area of the larynx', 'fluid will drain rapidly into the thorax below the vocal folds.', 'there are numerous mucous glands in the vestibular folds.', 'the vocal folds have a protective layer that prevents oedema', 'the larynx below the vocal folds is more susceptible to inflammation', 'the mucosa below the vocal folds absorbs fluid more rapidly', 'the mucosa covering the vocal folds is tightly attached to underlying tissues.', 'there are fewer blood vessels in the area below the vocal folds']\",\"question\":\"Laryngeal oedema is usually confined to the larynx above the vocal folds because\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}