{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"660671c8-0830-5ab2-baf2-44cc0d8b7fca","task_key":"default--test--3374","task_revision_id":"1","upstream_id":"3374","short_description":"A woman opens a can of ham and beans and suspects food poisoning . She boils the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The illness was caused by pre-formed toxins from Clostridium perfringens', \\\"The woman's food poisoning was likely caused by the toxin produced by Staphylococcus aureus.\\\", \\\"The woman's food poisoning was due to a virus\\\", 'The food poisoning was a result of chemical contamination from the can', \\\"The woman's symptoms were psychosomatic and not caused by the food\\\", 'The woman was poisoned by botulism', \\\"The woman's food poisoning was caused by a fungal infection\\\", 'The food poisoning was due to spoilage bacteria that were not destroyed by boiling', 'The symptoms were caused by an allergic reaction to an ingredient in the ham and beans', 'The nausea and vomiting were side effects of a medication the woman had taken, unrelated to the food']\",\"question\":\"A woman opens a can of ham and beans and suspects food poisoning . She boils the contents for half an hour. A few hours after eating the ham and beans, nausea, vomit-ing, and diarrhea set in. Explain.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}