{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be99af54-3ea7-5909-93d1-3ffed7dc0d82","task_key":"default--test--10410","task_revision_id":"1","upstream_id":"10410","short_description":"In the procedure Mystery below, the parameter number is a positive integer.","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['The procedure returns true when the initial value of number is a multiple of 2 or 3, and it otherwise returns false.', 'The procedure returns false when the initial value of number is greater than 2, and it otherwise returns true.', 'The procedure returns false when the initial value of number is a prime number, and it otherwise returns true.', 'The procedure returns true when the initial value of number is odd, and it otherwise returns false.', 'The procedure returns false when the initial value of number is even, and it otherwise returns true.', 'The procedure returns true when the initial value of number is less than 2, and it otherwise returns false.', 'The procedure returns true when the initial value of number is even, and it otherwise returns false.', 'The procedure returns true when the initial value of number is 2, and it otherwise returns false.', 'The procedure returns true when the initial value of number is a prime number, and it otherwise returns false.', 'The procedure returns true when the initial value of number is greater than 2, and it otherwise returns false.']\",\"question\":\"In the procedure Mystery below, the parameter number is a positive integer.\\n\\n PROCEDURE Mystery (number)\\n {\\n  REPEAT UNTIL (number <= 0)\\n  {\\n   number ← number - 2\\n  }\\n  IF (number = 0)\\n  {\\n   RETURN (true)\\n  }\\n  ELSE\\n  {\\n   RETURN (false)\\n  }\\n }\\n\\n Which of the following best describes the result of running the procedure Mystery?\",\"src\":\"ori_mmlu-high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}