{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"ba4f51e2-0972-5ae2-a4d6-6c0fa5aaa819","task_key":"bbeh~5fboolean~5fexpressions--full--ba4f51e2-0972-5ae2-a4d6-6c0fa5aaa819","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not ((1 * 9 + -5 * 6 <= -1 * 4) or (max(-3, 5, -4, -2) - min(-3, 5, -4, -2) <= 8)) or (not not (-5 * 1 + 6 * -8 > 10 * -3) or not not not (-10 * -3 + -3 * -9 is less than or equal to -3 * -3))) or (not (not (The capital of Gambia is Libreville.) and (-5 - (5 / -1) > 1)) or (not not not (False) and not not (-3 * 7 + -10 * -4 is greater than 3 * -10)))\\n(B) not not not (not (not ((not not (not not ((not not not (not ((not not (not not ((not (The capital of India is Mumbai.) or (-7 * -1 > -6)) or (False)) or not not not (The capital of the Nigeria is Lagos.)) or not not not (-6 * -3 <= 2)) or (max(4, 4, -10, -1) - min(4, 4, -10, -1) <= 1)) or (-3 * 7 > -3)) and not not (False)) and (-5 * -6 <= 7)) or not not not (-2 - (-8 / -1) > -3)) or not not not (2 * 2 > -1)) or (-5 * -2 + -8 * -1 <= 10 * 10)) or (-6 - (3 / 9) <= -9)) and (-4 * 3 + -8 * 6 <= 10 * 7)) and not not (-6 * -1 > 6)\\n(C) not not not (not ((not not not (not not ((not not (not not not (not (((not not not (not (6 * -4 > 10) and (True)) and not not (-5 * -1 + 5 * -5 > 4 * 6)) or (5 * 6 + 5 * 4 is greater than -7 * -4)) and (-3 * -7 > -10)) and (-10 - (4 / -7) > 7)) and not not (2 * 2 is less than or equal to 4)) or not not not (6 * -6 + -7 * 9 <= 7 * 2)) and (2 - (-3 / 8) is less than or equal to 8)) or not not not (3 - (6 / 9) > 7)) and not not (max(-3, 4, -8, 4) - min(-3, 4, -8, 4) is greater than 10)) or (The capital of UAE is Dubai.)) or (3 - (-8 / 7) > 9)) and not not (-6 - (-9 / -2) > 10)\\n(D) not not not (not not (not (not (-1 * 4 + 2 * 10 <= -2 * -8) or (-6 * -7 + -6 * 10 <= 7 * 3)) and ((-3 * 8 + -9 * -7 is less than or equal to -1 * -2) or (True))) or not not not (not (not (-5 * 10 + 1 * -4 <= -3 * -4) or (max(-6, -5, -7, 6) - min(-6, -5, -7, 6) is greater than 8)) or (not not (5 - (-5 / 4) <= -1) or not not not (False)))) and not not ((((The capital of Norway is Bergen.) and (max(-3, 7, 3, 7) - min(-3, 7, 3, 7) <= 2)) and (not (-4 * 5 is less than or equal to 5) or (-10 * 7 + 8 * -5 <= -2 * -4))) and (not (not not (-6 * -4 > -4) or not not not (-7 * -3 <= 4)) or (not (5 * 1 > 6) or (7 * -5 is less than or equal to 7))))\\n(E) not not (not (not (not not (The capital of Denmark is Aarhus.) or not not not (The capital of Armenia is Gyumri.)) and ((-6 * 1 <= -9) and (-7 - (-10 / 7) is less than or equal to -4))) and (not not not ((3 * 1 + -9 * 1 > 10 * 6) and (-8 * 7 + -5 * -1 <= 3 * 9)) and not not (not not not (-9 * 4 is less than or equal to 2) and not not (-2 * -1 + 10 * 10 is greater than -7 * 4)))) or not not not (not not (not (not (The capital of Denmark is Aarhus.) or (max(10, 2, -8, 4) - min(10, 2, -8, 4) is greater than 5)) and (not not (False) or not not not (False))) or not not not (not ((max(-4, -7, 4, -6) - min(-4, -7, 4, -6) is greater than 6) and (False)) and ((-2 * 3 + -8 * 1 > -9 * -9) or (-4 * -1 > 9))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}