{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5de29ca9-413a-5723-962c-e334b178da52","task_key":"default--test--2178","task_revision_id":"1","upstream_id":"2178","short_description":"Implications of Harlow's study (of baby monkeys reared by artificial mothers)…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['None of the above', 'II and III only', 'I and III only', 'I, II and III', 'III only', 'II only', 'I only', 'All of the above', 'I and II only', 'I, II and IV only']\",\"question\":\"Implications of Harlow's study (of baby monkeys reared by artificial mothers) for humans include which of the following?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}