{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f1bb143f-9480-518a-8a27-9f31757fd9ed","task_key":"bbeh~5fboolean~5fexpressions--full--f1bb143f-9480-518a-8a27-9f31757fd9ed","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not (not not not (not (The capital of Canada is Ottawa.) or (max(3, 4, -2, 7) - min(3, 4, -2, 7) <= 2)) and not not (not not (False) or not not not (-2 - (10 / 8) is less than or equal to -3))) or (not ((4 * -8 + -6 * -9 is less than or equal to -5 * 5) or (max(1, 8, 1, 8) - min(1, 8, 1, 8) is greater than 1)) or (not not not (-7 * 1 + -2 * 7 is less than or equal to -5 * 7) and not not (2 * 3 + -1 * -10 is less than or equal to -4 * 5)))) and (not not (not not not (not not not (True) and not not (max(1, -5, -2, 1) - min(1, -5, -2, 1) > 6)) and not not (not not not (8 * 3 + -5 * -3 is greater than 7 * 4) and not not (10 * 10 <= 2))) or not not not (not (not not not (max(-2, 3, -9, 10) - min(-2, 3, -9, 10) <= 4) and not not (The capital of Afghanistan is Kabul.)) or (not not (2 * 4 + -5 * -8 is greater than 3 * 4) or not not not (-1 * -4 <= 2))))\\n(B) (((not not not ((not (not (not (not (not (not not not (8 * 1 + 6 * -6 <= -4 * 2) and not not (7 - (10 / 10) is greater than 1)) or (max(-1, 8, 10, -7) - min(-1, 8, 10, -7) <= 4)) and (The capital of Iran is Isfahan.)) and (-4 * 9 is less than or equal to 10)) and (max(6, -7, 3, -4) - min(6, -7, 3, -4) is greater than 4)) or (max(10, -4, -2, -3) - min(10, -4, -2, -3) <= 5)) and (7 * -8 + -1 * 2 > -7 * 6)) and not not (9 * 10 + -6 * -5 is less than or equal to -6 * 9)) or (8 - (3 / 9) <= 7)) and (The capital of Armenia is Gyumri.)) and (-7 * -3 + 2 * 4 is greater than -3 * -1)\\n(C) not not not (not not (not not ((not (not (not (not not (not (False) and (False)) or not not not (The capital of Malaysia is Kuala Lumpur.)) or (6 * -6 is less than or equal to 3)) or (max(2, -7, 1, -3) - min(2, -7, 1, -3) is greater than 7)) or (-3 * -8 <= 9)) and (False)) or not not not (True)) or not not not (-3 * 7 is less than or equal to -10)) and not not (9 - (5 / -9) is greater than -5)\\n(D) (not ((not not not ((not (((((not (not (not not ((4 * -10 + 4 * -5 > -10 * 4) or (The capital of Jordan is Beirut.)) or not not not (1 - (4 / 9) > 8)) or (The capital of Gambia is Banjul.)) and (4 - (3 / -1) <= -7)) and (True)) and (10 - (9 / -10) is greater than -5)) or (True)) or (-4 * -4 + -7 * 5 > -9 * -3)) or (2 * 4 > 9)) or (-3 * -8 <= 2)) and not not (False)) or (-4 * 6 + -2 * 4 > 4 * -2)) or (False)) and (5 - (9 / 2) <= -10)\\n(E) ((not (not not not (not (not ((not (not not (not (not (not (max(-4, 7, 10, 2) - min(-4, 7, 10, 2) is greater than 5) or (max(-7, -5, 7, -6) - min(-7, -5, 7, -6) is greater than 3)) and (10 - (-4 / 7) <= -6)) or (-9 * -4 + -4 * -10 is greater than 1 * 8)) or not not not (-9 * 3 + 3 * -3 > -2 * 1)) or (max(3, -4, 4, 6) - min(3, -4, 4, 6) <= 1)) or (6 * -6 is less than or equal to 10)) or (-7 * 6 + 8 * -9 <= 9 * -3)) or (True)) and not not (max(-9, 8, 8, -5) - min(-9, 8, 8, -5) <= 6)) or (-8 - (8 / 9) is less than or equal to -8)) and (6 - (-8 / -7) <= 2)) and (5 * -7 + 4 * 2 > -8 * -3)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}