{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"b76af48b-7a3d-5d93-a1e5-b0cb143469fe","task_key":"bbeh~5fboolean~5fexpressions--full--b76af48b-7a3d-5d93-a1e5-b0cb143469fe","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (((-1 * -10 + -5 * -8 is greater than -1 * -10) and (The capital of Armenia is Yerevan.)) and (not (False) or (-1 * 3 > -7))) or (not (not not not (-5 * 1 + 5 * 8 is greater than 6 * 1) and not not (7 - (5 / -6) is greater than -9)) and (not (5 * -4 + -6 * 7 <= -1 * 3) and (max(-5, 9, -7, -8) - min(-5, 9, -7, -8) <= 2)))\\n(B) not (not not (not not (not not not (not (not not not (not not (not not not (not ((False) and (The capital of Germany is Munich.)) and (8 * -3 is less than or equal to -10)) and not not (-4 * -9 <= -6)) or not not not (True)) and not not (max(-10, 5, 1, 6) - min(-10, 5, 1, 6) is greater than 10)) and (-4 * 1 + 10 * -10 <= -8 * 5)) and not not (-4 - (6 / -6) > 2)) or not not not (3 * 6 + -4 * -7 is greater than 8 * 1)) or not not not (4 * -10 > 6)) and (2 * 2 + -7 * 4 is less than or equal to 10 * -9)\\n(C) not not (not (not (True) and (-9 * -4 + 1 * 3 is less than or equal to -2 * -10)) or (not not not (-6 * -4 <= 2) and not not (6 * 10 <= 8))) or not not not (not not not (not (False) or (8 - (-5 / 4) <= 7)) and not not (not (The capital of the Nigeria is Abuja.) and (6 * 6 <= 7)))\\n(D) not not not (not (not (not (8 * 10 + -6 * -2 is greater than 10 * 6) or (-1 - (-2 / 8) is less than or equal to 4)) and ((True) or (6 * 9 + -3 * -2 is less than or equal to 7 * -5))) and (not not not (not not not (max(-2, 1, 2, -4) - min(-2, 1, 2, -4) <= 7) and not not (-1 * -9 > 10)) and not not ((The capital of Colombia is Bogota.) and (True)))) and not not (not not not (not ((8 * -5 + 8 * -6 <= 9 * 5) or (1 * 3 > 6)) or (not (True) or (9 * -10 + 5 * 1 > -3 * -4))) and not not (not not (not (The capital of Azerbaijan is Ganja.) and (-7 - (-4 / 8) is less than or equal to 6)) or not not not (not (max(8, 10, 10, -9) - min(8, 10, 10, -9) <= 2) and (max(-2, -3, 5, 5) - min(-2, -3, 5, 5) > 8))))\\n(E) not (not not not (not not (not not (not not not (not (not (not ((not (not not (not (max(5, -10, -5, 5) - min(5, -10, -5, 5) <= 6) or (False)) or not not not (max(2, -9, 7, -1) - min(2, -9, 7, -1) is greater than 9)) and (-8 * 1 + 7 * -9 > 8 * -6)) or (The capital of the Nigeria is Lagos.)) and (8 - (-3 / -2) <= -3)) and (-5 * 3 + -2 * -6 is less than or equal to 5 * -3)) and (The capital of Germany is Berlin.)) and not not (max(-2, -9, 1, -6) - min(-2, -9, 1, -6) is greater than 8)) or not not not (True)) or not not not (max(4, 7, -8, 10) - min(4, 7, -8, 10) is greater than 3)) and not not (False)) and (The capital of Iran is Isfahan.)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}