{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"83af02e6-a5c6-517f-9410-646fe5d30d65","task_key":"bbeh~5fboolean~5fexpressions--full--83af02e6-a5c6-517f-9410-646fe5d30d65","task_revision_id":"3","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not (not (not not (not not (not not not ((-3 * -5 + -8 * 8 <= 3 * 10) and (max(4, 2, 9, -8) - min(4, 2, 9, -8) <= 1)) and not not (-10 - (4 / 10) is less than or equal to -7)) or not not not (-9 * -6 + 3 * -5 is greater than 4 * -7)) or not not not (max(-9, -4, -1, -9) - min(-9, -4, -1, -9) <= 9)) and (-8 * -2 > -1)) and (-3 * 3 + 2 * -5 <= -7 * -1)) and not not (max(9, 9, 10, 3) - min(9, 9, 10, 3) <= 7)\\n(B) not not not (not (not not (not (not not not (not not (not (not (not not (not (-8 * -9 + 7 * -5 is greater than 7 * 2) and (-7 * -8 + -5 * 8 is less than or equal to -2 * -5)) or not not not (4 * -1 is less than or equal to 5)) and (max(-5, 4, -6, 2) - min(-5, 4, -6, 2) > 10)) and (False)) or not not not (False)) and not not (max(-5, -8, -5, -2) - min(-5, -8, -5, -2) > 7)) or (max(-1, -10, 1, 9) - min(-1, -10, 1, 9) is greater than 10)) or not not not (max(-7, -8, 10, -5) - min(-7, -8, 10, -5) <= 10)) or (-2 * 4 > -4)) and not not (False)\\n(C) (not (not (not (-2 * -5 > -10) and (False)) or (not (-1 * -5 + 7 * 10 is less than or equal to 8 * -7) and (8 * 7 + 8 * -5 is less than or equal to 6 * -2))) and (not ((True) and (True)) or ((9 * -1 + 10 * 4 is greater than 2 * -4) or (9 * 9 + -2 * -9 is greater than 9 * -10)))) or (not (not not (not not not (max(1, -4, 5, -9) - min(1, -4, 5, -9) is greater than 7) and not not (8 * 6 > -10)) or not not not (not (1 - (4 / 10) is greater than -9) or (-8 - (10 / 7) is less than or equal to -8))) and (not not (not not (10 * -6 is less than or equal to 6) or not not not (The capital of Iran is Tehran.)) or not not not (not (4 * -4 > -7) and (max(-2, 10, -4, -2) - min(-2, 10, -4, -2) is greater than 7))))\\n(D) not not (((The capital of Azerbaijan is Ganja.) or (max(8, -9, -8, -7) - min(8, -9, -8, -7) <= 1)) and (not (The capital of India is Mumbai.) or (3 * -9 + -8 * -8 is less than or equal to 2 * -7))) or not not not (not (not (1 - (-9 / -3) > 1) and (-8 - (4 / -3) > -1)) or (not (-7 * -2 > 5) and (10 - (3 / 7) is greater than -4)))\\n(E) not (not not (not (True) or (True)) or not not not (not (-5 * 2 + 1 * -8 <= -1 * 5) and (max(9, 9, 9, -1) - min(9, 9, 9, -1) is greater than 4))) or (not (not (True) and (8 * -4 > 10)) and (not not not (6 * -7 is less than or equal to -8) and not not (max(3, -10, -9, -10) - min(3, -10, -9, -10) is greater than 3)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}