{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4264142-324a-556f-90ab-2c83e6746745","task_key":"default--test--2904","task_revision_id":"1","upstream_id":"2904","short_description":"Based on your knowledge of plant nutrition, explain how a photosynthetic plant…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['pale, small, short, and weak', 'Dark, small, and feeble', 'Purple, robust, and of average height', 'yellow, small, and feeble', 'Green, small, and sturdy', 'Pale, tall, and spindly', 'green, tall, and strong', 'Yellow, tall, and robust', 'Variegated leaves with normal growth', 'dark, large, and robust']\",\"question\":\"Based on your knowledge of plant nutrition, explain how a photosynthetic plant growing in iron - and magnesium - deficient soil would look.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}