{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a619aa69-4d8a-5ea0-93ac-08f3f153ca8a","task_key":"default--test--684","task_revision_id":"1","upstream_id":"684","short_description":"In the _________ stage in the adoption process, consumers become aware of the…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Persuasion', 'Awareness', 'Interest', 'Knowledge', 'Confirmation', 'Evaluation', 'Decision', 'Acceptance', 'Rejection', 'Implementation']\",\"question\":\" In the _________ stage in the adoption process, consumers become aware of the new proposition. They have little information and have yet to develop any particular attitudes towards the product. Indeed, at this stage consumers are not interested in finding out any more information.\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}