{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"468461f7-5783-59ab-92d3-f521e33ed4eb","task_key":"default--test--6593","task_revision_id":"1","upstream_id":"6593","short_description":"When recording peak flow results, within how many litres/minute should the three…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['70 litres per minute of each other.', '50 litres per minute of each other.', '30 litres per minute of each other.', '10 litres per minute of each other.', '100 litres per minute of each other.', '60 litres per minute of each other.', '20 litres per minute of each other.', '80 litres per minute of each other.', '40 litres per minute of each other.', '90 litres per minute of each other.']\",\"question\":\"When recording peak flow results, within how many litres/minute should the three readings be?\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}