{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"9a8dad08-5cd3-541d-8d07-795be06065f5","task_key":"bbeh~5fboolean~5fexpressions--full--9a8dad08-5cd3-541d-8d07-795be06065f5","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not not (not ((not not not ((not (not not (not not ((8 - (-9 / 2) is greater than -8) or (max(-7, -9, -2, -3) - min(-7, -9, -2, -3) <= 4)) or not not not (max(2, -3, -8, -5) - min(2, -3, -8, -5) <= 1)) or not not not (The capital of Norway is Bergen.)) and (-6 * 9 is less than or equal to -7)) and (False)) and not not (The capital of Belarus is Minsk.)) or (The capital of UAE is Dubai.)) and (10 * 4 + -6 * 9 is greater than 8 * -8)) or not not not (-2 * -7 + 7 * -2 > -9 * -6)) and (9 * 3 + 8 * -9 > 1 * -1)\\n(B) not not not ((not not not (True) and not not (True)) or (not (8 * 9 > 9) and (max(1, -7, -2, -4) - min(1, -7, -2, -4) > 9))) and not not (not not not (not not not (7 * -4 + -8 * -6 > -7 * -9) and not not (6 - (-8 / -10) is greater than -5)) and not not ((True) or (1 - (-2 / 2) > 5)))\\n(C) not not not (not (not not not (not (not ((not ((not (not not (not (False) or (7 * 7 <= -10)) or not not not (-1 * -7 + 3 * -10 > -2 * -3)) and (-9 * -10 + 10 * 8 is greater than -1 * 2)) and (10 * 5 + 1 * -6 is greater than -9 * 8)) and (10 * -5 + 4 * -1 <= 2 * 8)) or (3 * 8 + -6 * -6 is less than or equal to 9 * 2)) or (1 - (-6 / 1) <= -5)) and (True)) and not not (True)) and (10 - (8 / 7) is greater than 8)) and not not (True)\\n(D) not (not not (not (not not not (not ((not (not not (not not (-10 * -1 > 4) or not not not (False)) or not not not (False)) and (False)) or (4 - (-5 / 9) > 6)) and (3 - (-3 / -6) is greater than -8)) and not not (7 * -10 is less than or equal to 9)) or (True)) or not not not (-9 * -6 + -4 * 5 is greater than -5 * 1)) or (2 * -3 is less than or equal to 4)\\n(E) not (not not ((not not not (not (not (not (not not not (1 - (-9 / 5) is greater than -8) and not not (max(-3, 7, -6, 3) - min(-3, 7, -6, 3) <= 10)) or (6 * 1 <= 1)) or (1 - (8 / -6) is greater than -3)) and (False)) and not not (1 - (-9 / -8) > 9)) or (False)) or not not not (False)) or (max(-8, -3, 6, 5) - min(-8, -3, 6, 5) <= 4)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}