{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11453f96-2a1b-5ce6-a360-51e5d145f618","task_key":"all--test--11453f96-2a1b-5ce6-a360-51e5d145f618","task_revision_id":"1","upstream_id":"","short_description":"What role do women play in food security?","config":"all","split":"test","body":"{\"choices\":\"['Their main role is preparing meals.', 'They play a central role as producers of food, managers of natural resources, income earners and caretakers of household nutrition.', 'They weed the fields, but never plough or plant them, as only men are strong enough for those activities.', 'Their efforts focus mainly on vegetable gardens and chickens; men are involved with cereal crops and larger livestock.']\",\"question\":\"What role do women play in food security?\\n\",\"subject\":\"nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}