{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d548fca8-910f-510e-832e-1761373d4220","task_key":"default--test--10745","task_revision_id":"1","upstream_id":"10745","short_description":"You have just received the output from one of your Basic Assembly Language (BAL)…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Verify the input data used by the program', 'Trace the program execution step by step', 'Check the Cross Reference Table and the P/P Storage Dump', 'Check the syntax of the program', 'Recompile the program', 'Review the latest changes made to the code', 'Analyze the job control language (JCL) settings', 'Check the console log for runtime errors', 'Examine the Linkage Editor output', 'Check the logic flow of the program']\",\"question\":\"You have just received the output from one of your Basic Assembly Language (BAL) programs, but to your dismay, theoutput is not what you had hoped for. To debug this program, your first reaction is to check the contents of your storagelocations. What procedures would you follow inthis processof debugging?\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}