{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4562b716-76c5-54a5-8385-2f8d5691cf80","task_key":"default--test--10679","task_revision_id":"1","upstream_id":"10679","short_description":"What are the rules regarding the specification of a procedure?","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Procedures execute automatically during program execution', 'Procedures require a start statement but do not have a specific ending requirement; they terminate when the program execution completes.', 'Procedures can only be declared at the beginning of the program', 'Procedure blocks must be written in a separate file from the main program and are included during compilation.', 'Procedures can be declared with any number of names, and the END statement is optional, with the block ending at the first RETURN statement.', 'Procedures must be invoked at the start of the program and run continuously until the program ends.', 'Procedure blocks must be labeled with a unique number and can only contain variable declarations.', 'Procedures do not require an end statement', 'Procedures must be declared within the main function and cannot be invoked by other procedures or functions.', 'Procedure block may appear anywhere in the program, must start with a procedure statement with at least one name, must end with an END procedure-name statement, is bypassed during execution unless invoked by a procedure reference.']\",\"question\":\"What are the rules regarding the specification of a procedure?\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}