{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26060ac1-262b-5cbf-8d03-b02435ff5d8f","task_key":"default--test--10542","task_revision_id":"1","upstream_id":"10542","short_description":"How are tracers useful as a debugging aid?","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Tracers automatically correct errors in the source code', 'Tracers help to encrypt the program', 'Tracers compile the program into machine language', 'Tracers help to optimize the code', 'Tracers help to reduce the execution time of a program', 'Tracers predict future bugs in the program based on past trends', 'Tracers obfuscate the code to protect it from reverse engineering', 'Tracers increase the processing speed of the program', 'Tracers help locate bugs by analyzing and executing real machine instructions one at a time, allowing complete internal program inspection.', 'Tracers enhance the graphical user interface of the program']\",\"question\":\"How are tracers useful as a debugging aid?\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}