{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9baf44b6-119d-55d4-8ed1-ed96cc8da7c4","task_key":"default--test--6066","task_revision_id":"1","upstream_id":"6066","short_description":"The noroviruses typically cause which of the following?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['High fever and cough', 'Severe headache and muscle pain', 'Disease of the central nervous system', 'Loss of taste and smell', 'Projectile vomiting and diarrhoea', 'Rash', 'Rash and fever', 'Diarrhoea', 'Breathlessness and chest pain', 'Persistent hiccups and diarrhoea']\",\"question\":\"The noroviruses typically cause which of the following?\",\"src\":\"ori_mmlu-virology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}