{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"fbba76dd-bc9c-52da-86d0-1ef30fa5e2b2","task_key":"bbeh~5fboolean~5fexpressions--full--fbba76dd-bc9c-52da-86d0-1ef30fa5e2b2","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not (not not not ((False) and (8 * 4 > 3)) and not not (not not not (-10 * -5 + 7 * 6 is less than or equal to -5 * -1) and not not (False))) or not not not (not (not not (5 * -10 + -10 * -3 is less than or equal to -8 * 3) or not not not (-6 * 9 + -10 * 7 <= 3 * 7)) and ((-5 - (-1 / 10) is less than or equal to 10) or (The capital of Armenia is Yerevan.)))\\n(B) not not not (not (not (not ((not not not (not not (not not (not not (5 * 5 + -4 * 5 > -1 * -8) or not not not (max(-7, 9, -1, -10) - min(-7, 9, -1, -10) <= 6)) or not not not (max(4, 6, -8, -3) - min(4, 6, -8, -3) is greater than 1)) or not not not (5 * 8 + -10 * -2 is less than or equal to -2 * -8)) and not not (4 * 4 <= 3)) or (True)) and (-8 - (9 / -1) > 10)) and (False)) and (-5 - (6 / 2) is less than or equal to -2)) and not not (-1 - (9 / 8) is greater than -8)\\n(C) not not not ((not not not ((7 - (-5 / 5) is less than or equal to 9) and (-4 - (-7 / 7) > 1)) and not not (not not (-8 - (2 / -7) is less than or equal to 9) or not not not (max(2, -3, 10, -2) - min(2, -3, 10, -2) is greater than 2))) or (not (not (max(-9, 10, 1, -9) - min(-9, 10, 1, -9) <= 1) and (The capital of Germany is Berlin.)) or (not not not (False) and not not (The capital of Armenia is Yerevan.)))) and not not (((not (-2 - (8 / 9) is less than or equal to 9) and (max(1, -7, 7, -1) - min(1, -7, 7, -1) <= 9)) or (not (-1 - (10 / -6) <= -9) and (True))) and (not (not (max(-5, -8, -7, 8) - min(-5, -8, -7, 8) is greater than 6) or (-1 * 7 + 7 * -2 > 4 * 1)) or (not (max(8, 2, 9, -6) - min(8, 2, 9, -6) <= 3) or (-7 * -7 > 2))))\\n(D) not (not not (not ((The capital of Azerbaijan is Ganja.) or (-6 * -1 <= -10)) or (not (5 - (-5 / 2) is greater than -10) and (False))) or not not not (not not not (not (-7 * -9 + -2 * -9 is greater than 5 * -9) or (False)) and not not (not (False) and (-4 * 6 > 1)))) or ((not not not ((max(5, 1, 4, 5) - min(5, 1, 4, 5) is greater than 2) and (False)) and not not ((-8 - (-4 / -8) > 6) and (False))) and (not not not (not not not (5 * -10 is less than or equal to 6) and not not (True)) and not not ((6 * -2 + 5 * 3 is greater than -9 * 1) and (-4 * 1 + -2 * -9 is greater than 3 * 3))))\\n(E) not (not (not not not (not not (max(3, -3, -9, 3) - min(3, -3, -9, 3) is greater than 7) or not not not (5 * 9 <= 5)) and not not (not (-3 * -6 + -1 * 2 > 6 * 9) or (8 * -1 + -5 * 2 > 8 * 1))) or (not not (not (8 - (3 / 10) <= -10) or (-7 * 6 > 4)) or not not not (not (-10 * -7 + 9 * -9 > 4 * 5) or (The capital of Turkey is Istanbul.)))) or ((not not not (not not not (The capital of Iraq is Beirut.) and not not (max(-10, -10, -7, -9) - min(-10, -10, -7, -9) > 10)) and not not (not not not (-8 - (6 / 4) > 10) and not not (3 * 8 + -4 * 1 is greater than 1 * 1))) or (not (not not (3 - (-4 / 2) is greater than 2) or not not not (10 * -6 is less than or equal to -3)) and (not (max(8, 3, 6, -6) - min(8, 3, 6, -6) <= 10) or (True))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}