{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b7e9ee3-3b12-5fa8-924d-6aaa4ea0b79c","task_key":"default--test--6070","task_revision_id":"1","upstream_id":"6070","short_description":"The chance of acquiring HIV infection from factor VIII in the United States in…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The use of Factor VIII has been completely discontinued in the United States', 'Factor VIII is only distributed to hospitals with strict infection control protocols', 'Factor VIII is only used by individuals who are already HIV positive', 'Factor VIII is irradiated to kill any potential HIV', 'Factor VIII is important from countries with no reported HIV', 'Factor VIII is heat-treated to inactivate any potential HIV', 'All donors for factor VIII are screened', 'Factor VIII is made synthetically', 'Factor VIII is derived from animals, which cannot carry HIV', 'Factor VIII is subject to detergent “washing” prior to release']\",\"question\":\"The chance of acquiring HIV infection from factor VIII in the United States in 2011 is 0% primarily because:\",\"src\":\"ori_mmlu-virology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}