{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"123858ba-d84c-5b2d-b7a2-38ca20e3710a","task_key":"default--test--3067","task_revision_id":"1","upstream_id":"3067","short_description":"Why is the organism that causes smallpox not used in the vaccine against…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The organism is extinct in the wild and cannot be harvested for vaccine production', 'The organism that causes smallpox is too costly to use in vaccine production', 'The use of the actual virulent variola virus may be dangerous if too much is administered', 'Smallpox virus is confined to research laboratories', 'There is no specific treatment for smallpox', 'The smallpox virus mutates too rapidly to be effective in a vaccine', 'There are ethical concerns about using a live pathogen in vaccines', 'Vaccinia virus, a less dangerous relative, provides immunity without the risk of causing disease', 'The smallpox organism cannot be weakened or inactivated for safe vaccine use', 'Smallpox is never produced with the vaccinia virus']\",\"question\":\"Why is the organism that causes smallpox not used in the vaccine against smallpox?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}