{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ae68deb-4741-5ddf-aa9a-4f2b4a627147","task_key":"default--test--10708","task_revision_id":"1","upstream_id":"10708","short_description":"Explain what is known as the 'Count and/or Compare' methodof stopping a computer…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['A method where the computer job is stopped based on the occurrence of a specific error or set of errors', 'A method that relies on external signals or user intervention to stop the computer job', 'A method where the programmer sets a maximum limit to the amount of times a loop is carried out or a maximum limit to the value of the result of his program. When this maximum limit is reached, the computer is instructed to exit the loop and carry out the remainder of the program.', 'A method where the programmer sets a minimum limit to the number of user inputs before terminating the program', 'A method that uses a random number generator to determine when to stop the loop', 'A method where the computer stops the job when it detects that the system is idle', 'A method where the programmer allows the loop to run indefinitely', 'A method where the programmer sets a minimum limit to the loops', 'A method where the job is stopped when the program reaches a predetermined execution time limit', 'A method where the programmer sets a limit to the amount of memory used by the program']\",\"question\":\"Explain what is known as the 'Count and/or Compare' methodof stopping a computer job.\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}