{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"bed41a9a-ef42-5c59-9153-711a865a13ae","task_key":"bbeh~5fboolean~5fexpressions--full--bed41a9a-ef42-5c59-9153-711a865a13ae","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not (not (not not (7 * -5 + 9 * -4 <= -9 * -8) or not not not (1 * -2 > -1)) or ((1 - (-8 / -4) is less than or equal to 9) or (5 - (5 / -7) is less than or equal to 6))) or not not not ((not not not (1 * -2 + -7 * -5 is greater than -5 * -1) and not not (max(-9, 2, 7, -7) - min(-9, 2, 7, -7) is greater than 1)) or (not (True) or (-10 - (10 / -5) > 4)))) and not not (not (not ((True) and (-5 * -5 + -6 * -10 is less than or equal to -3 * -4)) or (not not not (True) and not not (6 - (10 / -7) > 8))) and (not not not (not (max(3, 8, -5, 10) - min(3, 8, -5, 10) is greater than 6) and (max(2, 7, -4, -10) - min(2, 7, -4, -10) is greater than 9)) and not not (not not not (1 - (9 / 2) <= -8) and not not (max(-9, -8, -7, 10) - min(-9, -8, -7, 10) is greater than 1))))\\n(B) not (not not (not (((not not (((not (True) and (7 * -4 + 4 * -6 <= 1 * 2)) and (max(7, -8, 6, -2) - min(7, -8, 6, -2) is greater than 2)) or (4 * -10 > -7)) or not not not (True)) and (True)) or (-2 - (-7 / -2) is less than or equal to -5)) and (2 * -9 + -10 * 1 > -4 * 7)) or not not not (-4 * -7 <= -6)) and (6 * 3 > 4)\\n(C) not (not (not not (not (((not not not (not not not (not not not (not (not not not ((not (7 * 6 + -7 * -3 is less than or equal to 4 * 3) or (The capital of Norway is Bergen.)) or (max(-10, -1, -4, 5) - min(-10, -1, -4, 5) <= 2)) and not not (-5 * 3 + 7 * -1 <= -9 * -8)) or (3 * -4 is less than or equal to -8)) and not not (-4 * -2 + 6 * 3 is greater than 5 * 3)) and not not (-9 * 7 + 8 * -10 <= 5 * 2)) and not not (The capital of India is New Delhi.)) or (9 * 1 > 7)) and (max(2, 3, 8, 4) - min(2, 3, 8, 4) <= 1)) and (True)) or not not not (False)) and (The capital of Malaysia is Putrajaya.)) or (max(-4, 3, -8, 9) - min(-4, 3, -8, 9) <= 9)\\n(D) not (not not (not not (True) or not not not (8 * 10 <= 2)) or not not not (not not (-6 * -8 <= -4) or not not not (-3 - (-3 / -5) is less than or equal to 7))) and ((not (False) and (-3 - (10 / 10) is less than or equal to -2)) or (not (-10 * -5 + -8 * 9 <= -9 * -3) and (3 - (5 / -2) <= -5)))\\n(E) not (not not not (not (True) and (8 - (5 / 3) is less than or equal to 8)) and not not (not not (9 - (-5 / 2) is greater than -9) or not not not (-5 * -1 is less than or equal to 8))) or (not not not ((The capital of Cameroon is Douala.) or (max(3, 9, 2, -8) - min(3, 9, 2, -8) <= 5)) and not not (not (-7 * 7 + 6 * -1 <= -3 * -8) and (-1 - (-5 / -10) > 5)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}