{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ff0de5f-bd9a-5726-a11f-fe86419da438","task_key":"default--test--10384","task_revision_id":"1","upstream_id":"10384","short_description":"Which of the following statements about circuits is (are) true?","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['II and III', 'I and III only', 'I and II only', 'I only', 'None of the above', 'II only', 'I, II, and III', 'All of the above', 'II and III only', 'III only']\",\"question\":\"Which of the following statements about circuits is (are) true?\\nI. Combinational circuits may have feedback; sequential circuits do not.\\nII. Combinational circuits have a \\\"memoryless\\\" property; sequential circuits do not.\\nIII. Both sequential and combinational circuits must be controlled by an external clock.\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}