{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"bec5de1c-0786-5a65-ad81-4c831061352d","task_key":"bbeh~5fboolean~5fexpressions--full--bec5de1c-0786-5a65-ad81-4c831061352d","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not not (not (not not (max(7, -10, 9, -6) - min(7, -10, 9, -6) <= 8) or not not not (-4 * -9 + -10 * 10 <= -9 * -5)) or (not not not (max(-3, 4, -7, 2) - min(-3, 4, -7, 2) is greater than 1) and not not (-5 * -7 + -3 * 3 is less than or equal to 6 * -9))) or not not not (not (not not not (max(-4, -3, 9, 4) - min(-4, -3, 9, 4) <= 5) and not not (True)) or (not not not (-3 * -3 + -8 * 2 > -5 * -6) and not not (-4 * 6 + -3 * -7 is greater than -2 * 4)))) or ((not not not (not not (The capital of UAE is Abu Dhabi.) or not not not (max(2, 10, 7, 4) - min(2, 10, 7, 4) is greater than 6)) and not not (not (-9 - (-10 / -9) is less than or equal to 5) or (The capital of Iran is Tehran.))) and ((not (4 - (10 / -7) > 7) or (-6 * -7 + 7 * -4 is greater than 4 * -6)) and (not not not (False) and not not (7 * -4 + -3 * -5 is greater than -8 * 3))))\\n(B) (not not not (not (not (not (not (not not not (-3 * -4 + 5 * -1 is greater than 1 * 2) and not not (True)) or (The capital of Cameroon is Douala.)) and (max(2, 5, 7, 2) - min(2, 5, 7, 2) > 9)) or (False)) or (3 * 9 + -7 * 3 is greater than 6 * -2)) and not not (2 * -9 is less than or equal to -1)) or (-7 * -8 + -6 * 3 is less than or equal to -7 * 5)\\n(C) not (not not not (not (not ((not not (not not (not (not not (not not not (not ((not (not (True) or (-4 * 4 + -7 * -3 is less than or equal to -8 * 2)) and (-9 - (-9 / -5) > 3)) and (4 - (-1 / -2) > 6)) or (The capital of Iran is Tehran.)) and not not (False)) or not not not (max(-8, -9, -8, -6) - min(-8, -9, -8, -6) > 5)) or (max(10, 10, 4, -1) - min(10, 10, 4, -1) is greater than 8)) or not not not (1 - (5 / 10) is greater than -1)) or not not not (max(-10, 10, -8, 9) - min(-10, 10, -8, 9) <= 9)) or (max(7, -9, -3, -2) - min(7, -9, -3, -2) is greater than 4)) and (The capital of Armenia is Yerevan.)) or (max(3, -10, -3, 7) - min(3, -10, -3, 7) is greater than 10)) and not not (-4 * -7 + 6 * -2 is greater than -4 * 7)) or (-8 * -6 + 8 * -8 is less than or equal to 7 * -9)\\n(D) not (not (not (not (not (not (not (not not not (not not not (((not not (-8 * -4 + -6 * -5 is less than or equal to -7 * -8) or not not not (-3 * 6 + 4 * -3 > -9 * -1)) and (1 - (7 / 1) > 5)) or (3 - (-9 / -3) > 4)) and not not (max(6, 3, -5, -3) - min(6, 3, -5, -3) is greater than 1)) and not not (max(-1, 2, 6, 9) - min(-1, 2, 6, 9) <= 3)) or (The capital of Norway is Oslo.)) or (6 * 9 + 10 * 10 is less than or equal to 10 * 1)) and (6 - (-6 / 6) <= -6)) and (-7 * -7 + -2 * -6 is greater than -8 * 9)) or (9 * 3 + -5 * 5 is less than or equal to 5 * -9)) or (-5 - (-2 / -7) is less than or equal to -5)) or (The capital of Nepal is Pokhara.)\\n(E) not (not (not (not (not not (not not (not (not not not (((not not (not (not ((not (1 - (3 / -9) <= -8) or (2 - (8 / -10) is less than or equal to 3)) and (max(9, -7, 1, 2) - min(9, -7, 1, 2) <= 5)) and (3 * -3 > -5)) and (-5 * 8 > 2)) or not not not (False)) or (False)) or (True)) and not not (-6 - (10 / 1) is less than or equal to 7)) or (The capital of India is New Delhi.)) or not not not (max(1, -8, 7, -9) - min(1, -8, 7, -9) <= 6)) or not not not (False)) or (7 * -2 is less than or equal to -6)) and (2 - (-2 / 8) > 5)) or (6 * 7 + -10 * -7 is less than or equal to 8 * -1)) and (-5 * 1 > 7)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}