{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab8c1f94-4471-574f-b787-e6a66a7a35da","task_key":"default--test--10680","task_revision_id":"1","upstream_id":"10680","short_description":"Define the term \"debugging.\"","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Debugging is the process of creating bugs.', \\\"Debugging refers to the enhancement of a program's performance without the need to fix faults.\\\", 'Debugging means deploying a program.', 'Debugging is the compilation of source code into executable programs.', 'Debugging is the process of testing a program.', 'Debugging involves the systematic elimination of a computer virus from a program.', 'Debugging is the initial design and development phase of a program.', 'Debugging is the process of optimizing the user interface of a program.', 'Debugging a program means the modification of programs, throughout its lifetime, in order to fix the faults that produce unacceptable or unexpected results.', 'Debugging is the routine updating of software to add new features.']\",\"question\":\"Define the term \\\"debugging.\\\"\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}