{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4a51568-0cc8-5e1a-8645-1da4e0f2edb9","task_key":"default--test--3038","task_revision_id":"1","upstream_id":"3038","short_description":"A woman opens a can of ham and beans and suspects food poisoning . She boils the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"[\\\"The woman was poisoned by a toxin that came from a food handler's skin sores.\\\", 'The woman was poisoned by a toxin that is destroyed by boiling.', 'The symptoms were due to psychological expectation of illness rather than actual food poisoning.', \\\"The woman was poisoned by a pesticide residue on the can's surface.\\\", \\\"The woman's symptoms were caused by an allergic reaction to an ingredient in the ham and beans.\\\", 'The woman was poisoned by the botulinus toxin.', 'The toxin produced by Staphylococcus aureus, which is heat-stable and unaffected by boiling, likely caused the poisoning.', 'The ham and beans were contaminated with E. coli, which was not eliminated by boiling.']\",\"question\":\"A woman opens a can of ham and beans and suspects food poisoning . She boils the contents for half an hour. A few hours after eating the ham and beans, nausea, vomit-ing, and diarrhea set in. Explain.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}