{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"07886ffa-77ea-5e5c-9800-b06ddf3c7f07","task_key":"bbeh~5fboolean~5fexpressions--full--07886ffa-77ea-5e5c-9800-b06ddf3c7f07","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not not (not (not not (-3 - (-5 / 3) > 10) or not not not (max(7, -4, -10, -9) - min(7, -4, -10, -9) <= 10)) or (not not not (max(-1, 5, 6, 9) - min(-1, 5, 6, 9) > 10) and not not (10 - (3 / -7) is greater than 1))) or not not not (not not not (not (False) and (False)) and not not (not (False) and (8 * 3 > 3)))) and (not not ((not not (max(-1, 6, -4, 6) - min(-1, 6, -4, 6) <= 6) or not not not (The capital of Armenia is Yerevan.)) and (not not (max(-10, -7, 10, -2) - min(-10, -7, 10, -2) <= 3) or not not not (-4 * 6 + -7 * -4 is greater than -10 * 2))) or not not not ((not not not (False) and not not (max(6, 2, 1, -3) - min(6, 2, 1, -3) is greater than 3)) or (not not (False) or not not not (-8 * 9 + -8 * -1 <= -10 * -1))))\\n(B) not not not (((not (not (not not not ((not (not ((max(-10, 9, 8, -7) - min(-10, 9, 8, -7) <= 6) or (False)) or (The capital of Canada is Ottawa.)) or (1 - (2 / -10) is greater than -5)) and (9 * 4 > 9)) and not not (True)) or (10 - (3 / 2) is less than or equal to 10)) or (5 * 7 <= -6)) or (-1 * 4 + -4 * 2 <= -2 * 6)) or (-5 - (9 / 9) > 5)) and not not (-1 * 10 + 3 * 10 is less than or equal to 5 * -10)\\n(C) not not (not ((True) or (True)) and (not not (False) or not not not (max(-1, 8, -4, -5) - min(-1, 8, -4, -5) <= 3))) or not not not ((not (False) or (8 - (-6 / -1) is less than or equal to 5)) or (not not not (False) and not not (-1 - (-6 / 1) > 9)))\\n(D) not not not (((not ((not not not (((not (not (not (The capital of Armenia is Yerevan.) and (max(-4, -4, -8, -6) - min(-4, -4, -8, -6) is greater than 1)) and (The capital of Colombia is Bogota.)) and (The capital of Afghanistan is Kabul.)) or (-3 * 6 > 6)) and (The capital of the Nigeria is Lagos.)) and not not (True)) or (True)) or (5 * -10 > -2)) and (4 - (-8 / -6) is less than or equal to 4)) and (7 * -9 + 9 * 6 > -4 * -8)) and not not (The capital of Afghanistan is Kabul.)\\n(E) not (not not not ((True) or (7 - (-7 / -9) > 8)) and not not (not not (4 * -3 + -6 * 6 is less than or equal to 10 * -7) or not not not (False))) and (not not not ((-5 - (2 / 6) > 10) or (-2 - (-5 / 2) is greater than -5)) and not not (not (max(-2, 1, 7, -5) - min(-2, 1, 7, -5) is greater than 5) and (False)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}