{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01735b73-422b-55b6-96fb-dbea82dfe144","task_key":"default--test--361","task_revision_id":"1","upstream_id":"361","short_description":"The focus on issues, the intended audience extending beyond shareholders and the…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Social accounting, Environmental management initiatives.', 'Financial accounting, Environmental management initiatives', 'Social accounting, Annual report publication.', 'Annual report publication, Social accounting', 'Environmental management initiatives, Social accounting.', 'Environmental management initiatives, Financial accounting.', 'Annual report publication, Financial accounting.', 'Annual report publication, Environmental management initiatives.', 'Social accounting, Financial accounting.', 'Financial accounting, Social accounting.']\",\"question\":\"The focus on issues, the intended audience extending beyond shareholders and the voluntary nature of ___________ distinguish it from _________.\",\"src\":\"ori_mmlu-business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}