{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0cbd27c5-c241-56bb-8ce3-d68c88f6771f","task_key":"default--test--2969","task_revision_id":"1","upstream_id":"2969","short_description":"We know that radiation and chemicals can cause cancer. Can viruses be a cause of…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Only radiation can cause cancer', 'Only chemicals can cause cancer', 'Only inherited genetics can cause cancer', 'Only lifestyle factors such as diet and exercise can cause cancer', 'Yes, viruses can be a cause of cancer', 'No, viruses cannot cause cancer', 'Viruses can increase the risk but are not a direct cause of cancer', 'Yes, but only bacteria, not viruses, can be a cause of cancer']\",\"question\":\"We know that radiation and chemicals can cause cancer. Can viruses be a cause of cancer?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}