{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0ca602db-a279-5132-8374-916655e92176","task_key":"default--test--2560","task_revision_id":"1","upstream_id":"2560","short_description":"As psychologists began to study the effects of mass communication, they saw the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The distinction is dependent on which view you choose to take', 'Propaganda disseminates false information, while education provides only true information', 'Education is designed for individual learning, while propaganda targets groups', 'Propaganda is always biased', 'Propaganda is a form of education that focuses on cultural knowledge', 'Education is always unbiased', 'Education aims to inform, while propaganda aims to persuade', 'Education and propaganda are the same thing', 'Propaganda uses only emotional appeals, while education relies solely on facts', 'Education is a government tool, while propaganda is used by private organizations']\",\"question\":\"As psychologists began to study the effects of mass communication, they saw the need to distinguish between education and propaganda. Is there a distinction, and if so, what is it?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}