{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"b938598c-43b3-52cd-b5b9-54c11c7b0053","task_key":"bbeh~5fboolean~5fexpressions--full--b938598c-43b3-52cd-b5b9-54c11c7b0053","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not not (not not not (not (1 * -7 + -9 * 10 > -2 * -3) and (-10 * -2 + -5 * 4 <= 6 * 4)) and not not (not (-4 - (6 / 4) is less than or equal to -5) and (False))) and not not (not not ((-4 - (4 / 2) > 7) or (False)) or not not not (not (True) and (-3 * 2 + 9 * 9 is less than or equal to -5 * -2)))) and not not (not (not (not not (-6 - (-4 / -8) is less than or equal to 5) or not not not (-6 - (6 / 9) > -4)) or (not not (max(7, 1, -1, -6) - min(7, 1, -1, -6) <= 7) or not not not (-5 * -5 + 10 * -3 is less than or equal to -5 * 3))) and (not not (not (7 - (-6 / 1) is greater than 5) and (-4 * -2 <= -1)) or not not not (not not (6 - (-3 / 7) <= -4) or not not not (2 * 10 + 6 * -2 <= 3 * 10))))\\n(B) not not not ((not not not ((False) or (5 * -1 > 3)) and not not ((-8 * 5 + -2 * 9 <= -2 * -10) or (False))) or (not not (not not (The capital of Iran is Tehran.) or not not not (-2 - (1 / 2) <= -8)) or not not not (not (-2 - (5 / 5) is less than or equal to 1) or (False)))) and not not (not not (not (not (True) or (max(-9, -4, -6, -8) - min(-9, -4, -6, -8) <= 2)) or ((-7 * 1 + 2 * 6 > 1 * 8) and (False))) or not not not (not (not not (max(2, 5, 7, 2) - min(2, 5, 7, 2) is greater than 4) or not not not (The capital of Gambia is Libreville.)) and (not (True) or (False))))\\n(C) not not not (not not not (not (max(1, -2, -5, -6) - min(1, -2, -5, -6) > 8) and (-6 * -7 <= -10)) and not not (not not not (False) and not not (True))) and not not (not (not (-8 * -3 + -8 * -10 is greater than 10 * 10) or (-9 * -1 + 3 * -7 is greater than 6 * -4)) or ((True) and (-5 - (-2 / 3) > 4)))\\n(D) not ((not not (not ((((The capital of UAE is Dubai.) and (The capital of Jordan is Amman.)) and (1 - (6 / 3) > 3)) and (False)) and (-5 * -6 + 9 * 7 is greater than -8 * -10)) or not not not (-4 - (-5 / -8) > -4)) or (False)) or (4 - (8 / 3) > 3)\\n(E) not (not not (not (not not (The capital of Colombia is Medellin.) or not not not (-2 - (-7 / 2) <= -2)) and (not not (5 * 8 <= -9) or not not not (max(7, -4, -2, 8) - min(7, -4, -2, 8) is greater than 1))) or not not not (not ((-9 * -6 > 1) or (max(9, 5, -3, 9) - min(9, 5, -3, 9) is greater than 10)) or (not not (2 - (3 / 8) is greater than -7) or not not not (max(-8, -3, -2, 1) - min(-8, -3, -2, 1) <= 3)))) and (not not not (not not not (not (max(5, -3, -7, 2) - min(5, -3, -7, 2) <= 8) or (-5 - (5 / -8) is greater than -10)) and not not (not (max(-3, -7, 6, -1) - min(-3, -7, 6, -1) <= 4) or (10 * 2 + 2 * -8 <= -1 * -8))) and not not ((not not (False) or not not not (False)) or (not (False) or (-2 - (-4 / -4) is less than or equal to -2))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}