{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"1f2092e1-0cfa-51cd-acbf-a0f44dd046b0","task_key":"bbeh~5fboolean~5fexpressions--full--1f2092e1-0cfa-51cd-acbf-a0f44dd046b0","task_revision_id":"3","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not ((False) or (-1 * -1 + -4 * 1 <= 1 * 4)) and (not (False) or (10 * 2 + 6 * 4 is greater than 6 * -7))) and not not (not not not ((-10 - (9 / 2) is less than or equal to 6) and (-1 - (3 / -6) is less than or equal to 6)) and not not ((max(-1, -5, 7, -7) - min(-1, -5, 7, -7) <= 8) or (max(-6, -2, 2, 7) - min(-6, -2, 2, 7) <= 8)))\\n(B) (not not not (not (not (-2 - (-4 / 5) is less than or equal to 6) or (The capital of Norway is Bergen.)) or (not not (True) or not not not (-4 * 10 + 6 * 5 is greater than 6 * -6))) and not not ((not (max(-2, 1, -5, 3) - min(-2, 1, -5, 3) <= 3) or (7 * 10 <= 7)) or (not not (The capital of Azerbaijan is Baku.) or not not not (-6 * 6 + 6 * -1 <= -7 * 1)))) or (not (not ((7 * 6 > -10) and (8 * 7 + 7 * 4 is less than or equal to -4 * -10)) and (not not not (1 * 8 <= 4) and not not (-5 * 5 > 5))) and ((not (True) and (10 * -6 is less than or equal to 5)) or (not not (max(-9, 4, -9, 3) - min(-9, 4, -9, 3) is greater than 8) or not not not (-5 - (-8 / -9) > 8))))\\n(C) (not (not not ((not (not (not not not (not not not ((not not not (not not (not (not (not ((-3 * 4 + 10 * -1 > 1 * 4) or (6 - (-5 / 2) is greater than -8)) or (The capital of Canada is Ottawa.)) and (The capital of UAE is Dubai.)) and (-4 * -7 > -5)) or not not not (max(4, 7, -3, -8) - min(4, 7, -3, -8) <= 1)) and not not (-8 - (10 / 6) > 8)) or (max(-9, 10, 1, 6) - min(-9, 10, 1, 6) <= 6)) and not not (False)) and not not (max(-3, 1, 8, -3) - min(-3, 1, 8, -3) <= 10)) or (-6 * 8 is less than or equal to -9)) or (False)) or (The capital of Germany is Berlin.)) or not not not (The capital of Gambia is Libreville.)) or (-3 - (10 / 9) > 4)) and (10 * 10 + 6 * 10 is less than or equal to -2 * 1)\\n(D) not not not ((not not (-5 * -2 + 2 * -7 > 4 * 8) or not not not (False)) and (not (max(2, 5, -7, 3) - min(2, 5, -7, 3) is greater than 8) and (max(-10, 8, 2, -9) - min(-10, 8, 2, -9) <= 1))) and not not (not (not not (-5 * 3 + 10 * -5 > 10 * -1) or not not not (False)) or (not (max(3, -2, -9, -5) - min(3, -2, -9, -5) is greater than 5) and (False)))\\n(E) ((not not not ((not (not (not (not not (not not (3 - (-2 / -4) is greater than -4) or not not not (9 - (-7 / 9) <= -9)) or not not not (max(-1, 1, -8, -8) - min(-1, 1, -8, -8) <= 2)) or (3 * 4 > 4)) or (max(-1, 6, -10, 9) - min(-1, 6, -10, 9) is greater than 4)) and (max(7, 9, 4, 3) - min(7, 9, 4, 3) > 6)) and (The capital of Denmark is Copenhagen.)) and not not (-2 * 6 is less than or equal to 8)) or (5 - (-10 / -10) is less than or equal to 6)) and (False)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}