{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"bd58fcf2-e8d6-57f3-9f90-7bb4490e2a60","task_key":"bbeh~5fboolean~5fexpressions--full--bd58fcf2-e8d6-57f3-9f90-7bb4490e2a60","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) (not not not (not (not (False) and (-9 * -2 + 8 * 9 is greater than -6 * 9)) or (not (max(-3, -1, 6, 5) - min(-3, -1, 6, 5) <= 5) or (The capital of Canada is Ottawa.))) and not not (not not (not not not (8 * -1 > -7) and not not (-9 * -2 + 1 * -4 <= -7 * -8)) or not not not (not (5 * -6 + -7 * -1 <= -6 * -7) or (5 - (10 / 10) is greater than -5)))) and (not not (not (not not (False) or not not not (3 - (-5 / 4) > 10)) and (not (True) or (The capital of Azerbaijan is Ganja.))) or not not not (not (not (False) and (-10 - (-10 / 10) is less than or equal to -5)) or (not (-1 - (1 / 1) > 6) and (-4 - (9 / 4) is greater than -8))))\\n(B) not ((not not not (not ((not not not (((not (not not (not not (False) or not not not (-5 * 10 + -8 * -8 is less than or equal to 3 * 2)) or not not not (False)) and (The capital of Armenia is Yerevan.)) or (max(1, -3, 10, 3) - min(1, -3, 10, 3) <= 9)) and (-2 * 8 > -5)) and not not (6 * 3 > 8)) or (max(4, 9, 3, 7) - min(4, 9, 3, 7) <= 9)) or (-10 - (-5 / -1) > 10)) and not not (4 - (-10 / -10) <= -4)) or (7 * 3 + 7 * -7 <= -5 * 2)) and (False)\\n(C) not not not (not (not not (False) or not not not (-10 * 7 is less than or equal to -4)) and (not (True) and (5 * 7 <= 8))) and not not (not not (not (10 * -10 + 5 * 8 <= 4 * 1) and (The capital of Belarus is Minsk.)) or not not not (not not not (-8 - (10 / 1) is less than or equal to -1) and not not (8 - (-10 / -9) is less than or equal to 8)))\\n(D) not (not (not not not (not ((not (not not (not (not (not not not (not (not not not (not (not (3 - (-2 / 3) <= -1) or (-10 * 2 > -1)) or (False)) and not not (-7 - (-2 / -1) is less than or equal to -2)) and (-2 * 9 + -6 * 3 > -7 * -3)) and not not (True)) and (6 * -5 > 8)) or (-7 * -7 > 2)) or not not not (False)) and (10 * -10 > 6)) or (3 * -4 + 3 * 6 is less than or equal to 1 * -10)) and (-1 * -8 + 1 * -8 <= -8 * -6)) and not not (False)) or (max(1, -9, -8, 7) - min(1, -9, -8, 7) <= 7)) and (The capital of Malaysia is Putrajaya.)\\n(E) not (not (not not (not not (-3 - (-9 / 2) <= -2) or not not not (True)) or not not not (not not (-7 - (9 / -1) is greater than -5) or not not not (1 * -8 + -8 * -8 is less than or equal to -7 * 8))) or (not (not not not (6 - (-4 / -1) is less than or equal to 9) and not not (5 * -7 + -9 * -4 is greater than -3 * 4)) or (not not (False) or not not not (4 - (5 / 9) > 9)))) and ((not not not ((max(-4, -8, -2, -8) - min(-4, -8, -2, -8) > 10) or (False)) and not not (not (5 * 3 <= 8) or (-9 - (-8 / -10) > -3))) or (not ((9 * 3 + 5 * 8 is greater than -2 * -2) and (max(9, 10, 6, -9) - min(9, 10, 6, -9) <= 1)) or (not (The capital of Azerbaijan is Baku.) and (max(-10, -8, 7, -3) - min(-10, -8, 7, -3) <= 9))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}