{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56272c9a-1fd5-555c-b9ea-103097b2b6d7","task_key":"default--test--3480","task_revision_id":"1","upstream_id":"3480","short_description":"A woman opens a can of ham and beans and suspects food poisoning . She boils the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The woman was poisoned by Clostridium botulinum toxin.', 'The woman was allergic to an ingredient in the ham and beans.', 'The can of ham and beans was contaminated with E. coli bacteria.', 'The nausea, vomiting, and diarrhea were caused by a virus unrelated to the ham and beans.', 'The woman had a normal food reaction.', 'The woman was poisoned by a pesticide residue in the ham and beans.', 'The woman was poisoned by Salmonella bacteria present in the ham and beans.', 'The food was not contaminated.', 'The woman was probably poisoned by the toxin produced by Staphylococcus aureus.', 'The symptoms are a result of psychological factors rather than food poisoning.']\",\"question\":\"A woman opens a can of ham and beans and suspects food poisoning . She boils the contents for half an hour. A few hours after eating the ham and beans, nausea, vomit-ing, and diarrhea set in. Explain.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}