{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d173efa8-f034-563b-9e2c-7ea84dcf55e1","task_key":"default--test--6121","task_revision_id":"1","upstream_id":"6121","short_description":"It has been observed that a reduction in family size and improved sanitary…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Living on a working farm will increase the development of atopic diseases', 'Particular care must be taken during breast feeding in high risk infants', 'Living on a working farm will help reducing the development of atopic diseases', 'Using antibiotics frequently will reduce allergies', 'Living in urban areas will help reduce the development of atopic diseases', 'The frequent use of antiseptic swipes will reduce allergies', 'Exposure to allergens in early childhood will increase the risk of developing allergies', 'Staying in highly sanitized environments will reduce the development of atopic diseases', 'Breastfeeding should be avoided in high risk infants to prevent allergies', 'The use of probiotics in infants will increase the development of atopic diseases']\",\"question\":\"It has been observed that a reduction in family size and improved sanitary condition have led to an increase in allergic conditions. The 'hygiene hypothesis' suggests that\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}