{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2aee9faa-2f32-5513-81c0-2a807a240e1b","task_key":"default--test--10438","task_revision_id":"1","upstream_id":"10438","short_description":"State the basic laws of Boolean algebra.","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"[\\\"The basic laws of Boolean algebra are Idempotent, Commutative, Associative, Distributive, Law of Union, Law of Intersection, Double Negative Law or Involution Law, Law of complement and deMorgan's law\\\", 'The basic laws of Boolean algebra include only Addition, Multiplication, and Complement', 'The basic laws of Boolean algebra are Scalar Multiplication, Vector Addition, Dot Product, and Cross Product', 'The basic laws of Boolean algebra are Associative, Distributive, Commutative, and Identity', 'The basic laws of Boolean algebra consist of the Law of Excluded Middle, Law of Non-Contradiction, and Law of Identity', 'The basic laws of Boolean algebra are Idempotent, Non-Commutative, Associative, and Exclusive OR', 'The basic laws of Boolean algebra are Commutative, Associative, Distributive, Identity and Inverse', 'The basic laws of Boolean algebra are Transitive, Reflexive, Symmetric, and Inverse', 'The basic laws of Boolean algebra are Reduction, Division, Integration, and Differentiation', 'The basic laws of Boolean algebra are Addition, Subtraction, Multiplication and Division']\",\"question\":\"State the basic laws of Boolean algebra.\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}