{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25a8c4df-c5ac-53b5-8ea6-a70b50aa04cf","task_key":"default--test--3146","task_revision_id":"1","upstream_id":"3146","short_description":"\"A virulent organism is as good as dead if it is not communicable .\" Explain.","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Pathogens can survive indefinitely on surfaces without a host.', 'Pathogens evolve to be less virulent in order to coexist with the host without transmission.', 'Pathogens are most successful when they cause chronic, non-lethal infections in a single host.', 'Pathogens require sunlight to remain virulent and communicable.', \\\"A pathogen's virulence is unrelated to its ability to spread from host to host.\\\", 'Pathogens are incapable of surviving outside a host.', 'The success of a pathogen depends on its successful transmittance to a new host via air, food, water, insects, or by contact.', \\\"Pathogens are always killed by the host's immune system.\\\", 'Pathogens can only reproduce in aquatic environments.', 'Pathogens can only survive within their original host.']\",\"question\":\"\\\"A virulent organism is as good as dead if it is not communicable .\\\" Explain.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}