{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ba8649e-0da6-5acc-96c2-93e390d6e44d","task_key":"default--test--5918","task_revision_id":"1","upstream_id":"5918","short_description":"An auditor decides to use the blank form of positive accounts receivable…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The auditor may have selected the wrong accounts to send the blank form to.', 'The blank form might be too complicated for the recipient to understand.', 'Accounts actually confirmed may not be representative of the population.', 'Accounts unconfirmed may have already been written off as uncollectible.', 'The blank form might be lost in the mail, resulting in no response.', 'All accounts do not have an equal opportunity to be selected for confirmation.', 'The recipient may not have the necessary information to fill the blank form.', 'Few responses may occur because more effort is required of recipients.', 'The blank form may be mistaken for junk mail and discarded.']\",\"question\":\"An auditor decides to use the blank form of positive accounts receivable confirmation. The auditor should be aware that the blank form may be ineffective because\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}