{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b19ec881-c512-57d9-8a7e-8b8eee0bb6a2","task_key":"all--test--b19ec881-c512-57d9-8a7e-8b8eee0bb6a2","task_revision_id":"1","upstream_id":"","short_description":"Although theoretically any food protein could act as a food allergen, there are…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Cow's milk, egg\\\", 'Mustard seeds, lamb,', 'Soy, peanuts', 'Fish, shellfish']\",\"question\":\"Although theoretically any food protein could act as a food allergen, there are eight major food allergens in Europe and North America which account for most of the allergic triggers observed in childhood. Which of the allergens pairings contains an allergen, which is not part of the group of eight major allergens?\\n\",\"subject\":\"nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}