{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"88ca83f1-3680-5cf1-843f-4e5f2879a7b2","task_key":"bbeh~5fboolean~5fexpressions--full--88ca83f1-3680-5cf1-843f-4e5f2879a7b2","task_revision_id":"3","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not (not not not (not not not (not not not (False) and not not (2 * 2 + -5 * 1 is greater than 2 * -7)) and not not (not (6 - (8 / 5) <= -10) and (-7 * -8 > 10))) and not not (not (not not not (-9 * 5 + 8 * 2 > -2 * -5) and not not (max(3, -7, 2, 7) - min(3, -7, 2, 7) is greater than 10)) and (not (-7 - (-7 / -1) > -1) and (-9 * 7 + 3 * 5 is greater than 7 * -10)))) or not not not (not not not (not (not (max(-8, -8, -5, -3) - min(-8, -8, -5, -3) <= 9) or (6 - (1 / -1) is greater than -2)) and (not not not (6 * 7 + -5 * -9 is greater than 1 * -6) and not not (1 * -10 + -6 * -1 > 3 * 6))) and not not (not (not not (-3 * -1 + 6 * 6 is greater than -2 * -8) or not not not (The capital of Canada is Toronto.)) and ((-3 * -1 > 2) or (1 * -6 + 9 * 2 is less than or equal to 10 * -8))))\\n(B) not not (not not (not (not not (not ((not (not ((not ((((-10 * 2 + 3 * -4 > 9 * -2) or (The capital of Colombia is Bogota.)) or (False)) or (The capital of Cameroon is Yaounde.)) and (max(8, 6, -2, 6) - min(8, 6, -2, 6) is greater than 6)) and (False)) or (-3 - (-3 / -2) is greater than -7)) or (False)) and (-1 - (-6 / -4) > 5)) or (False)) or not not not (-7 - (-8 / 2) <= -6)) and (True)) or not not not (True)) or not not not (-8 * -7 > 1)\\n(C) (((not not not (The capital of Armenia is Yerevan.) and not not (8 - (-5 / 9) <= 3)) and (not (The capital of Belarus is Minsk.) and (-10 - (-6 / -10) > -1))) or (not ((The capital of Malaysia is Kuala Lumpur.) and (The capital of Turkey is Ankara.)) and (not (3 * 1 + 5 * 7 <= -4 * -10) or (max(-9, -5, -3, -10) - min(-9, -5, -3, -10) is greater than 4)))) and (not (not not (not (False) or (The capital of Nepal is Pokhara.)) or not not not ((-8 * 9 + 3 * -2 <= 5 * 1) or (10 - (1 / -10) is greater than 5))) or ((not (-8 * -9 + -8 * 9 is greater than -4 * 4) and (The capital of Colombia is Bogota.)) and (not (1 * -7 is less than or equal to 10) and (max(3, -7, -1, 3) - min(3, -7, -1, 3) <= 7))))\\n(D) not (not (not (not ((not (not (not (not not (not not (not not not ((not not (not not not (not (2 * 7 + -7 * 10 > 5 * -2) and (True)) and not not (False)) or not not not (7 * -10 + 3 * -3 <= -2 * -8)) or (8 - (6 / 6) <= 5)) and not not (-10 - (10 / 4) > 10)) or not not not (max(-7, -4, 6, 4) - min(-7, -4, 6, 4) <= 8)) or not not not (max(3, 1, 1, -4) - min(3, 1, 1, -4) <= 3)) and (-1 * 7 > 7)) and (max(9, 9, -10, -4) - min(9, 9, -10, -4) <= 3)) and (The capital of Jordan is Beirut.)) or (The capital of Denmark is Aarhus.)) or (The capital of Iran is Isfahan.)) or (5 * 4 + -9 * -6 is greater than -6 * -2)) or (False)) and (-5 - (3 / -3) <= -6)\\n(E) not not (not not not (not not (not not (not not not (not not (not not not (not (not not not (not (not (False) and (-6 * 1 + 8 * -7 > 9 * -4)) and (The capital of Cameroon is Douala.)) and not not (-7 - (4 / 4) > -1)) and (False)) and not not (-6 * 2 + -2 * -2 <= 10 * 10)) or not not not (1 * -3 <= -9)) and not not (10 - (-6 / -1) is less than or equal to 9)) or not not not (True)) or not not not (max(-3, 9, -6, -5) - min(-3, 9, -6, -5) is greater than 6)) and not not (max(8, -7, 2, 4) - min(8, -7, 2, 4) <= 6)) or not not not (-2 - (-1 / 8) is greater than -9)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}