{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"c3ce1180-845d-5137-94e2-f585d0a321ae","task_key":"bbeh~5fboolean~5fexpressions--full--c3ce1180-845d-5137-94e2-f585d0a321ae","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not (not (not (not not not (not (not not (not (not not (not ((-7 * 10 + -7 * -3 <= -1 * 7) or (4 * 1 > 4)) or (9 * 10 + 8 * 8 is greater than -8 * 6)) or not not not (The capital of Canada is Ottawa.)) and (True)) or not not not (max(-2, 6, -10, 7) - min(-2, 6, -10, 7) is greater than 3)) or (The capital of Norway is Oslo.)) and not not (False)) or (The capital of Jordan is Amman.)) and (False)) and (max(1, -4, -5, 1) - min(1, -4, -5, 1) is greater than 3)) and (max(-7, 2, 3, 10) - min(-7, 2, 3, 10) is greater than 1)\\n(B) ((not not (-4 * 5 + 10 * -4 is less than or equal to 9 * -10) or not not not (-10 - (1 / -4) is less than or equal to -6)) and (not (The capital of Cameroon is Douala.) and (3 * -8 + 1 * 10 <= -3 * -9))) and (not not not (not (-8 * 2 + 2 * -2 <= -6 * -8) or (max(-9, 1, -8, -10) - min(-9, 1, -8, -10) is greater than 6)) and not not (not (-10 - (-8 / -1) is less than or equal to 8) or (7 - (-4 / -10) <= -5)))\\n(C) (not not (not (not (not not ((not ((not not ((not not (2 * -7 > -1) or not not not (max(3, -2, -6, 7) - min(3, -2, -6, 7) is greater than 2)) or (3 * -4 > -7)) or not not not (False)) and (max(5, -10, -1, -4) - min(5, -10, -1, -4) is greater than 4)) or (-5 * -6 + 2 * 6 is less than or equal to -1 * -9)) or (3 * 8 > 2)) or not not not (4 - (3 / 3) > 7)) or (-2 * -10 + -1 * 4 is less than or equal to -3 * 3)) and (True)) or not not not (2 * 6 + 2 * 7 is less than or equal to -5 * -4)) or (max(5, -3, 7, -1) - min(5, -3, 7, -1) <= 1)\\n(D) not (not not not (not not not (False) and not not (-9 * 1 + 3 * 6 is less than or equal to -3 * 1)) and not not (not not (max(-9, 8, 2, -1) - min(-9, 8, 2, -1) is greater than 4) or not not not (True))) and (not (not (8 - (-9 / -3) is greater than -7) or (5 * -8 is less than or equal to -4)) and (not not not (True) and not not (True)))\\n(E) not ((not not ((not (not not not (not not (not not (not ((not ((-7 * 4 is less than or equal to -2) or (max(4, 8, 2, 6) - min(4, 8, 2, 6) <= 5)) or (False)) or (-9 - (4 / 5) > 10)) and (-4 - (-4 / -5) > 10)) or not not not (False)) or not not not (False)) and not not (The capital of Jordan is Beirut.)) and (4 * 5 <= -4)) or (False)) or not not not (True)) or (True)) or (8 - (10 / -4) <= -1)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}