{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"b8dedc0a-4d0b-51f4-b612-1ae854aab368","task_key":"bbeh~5fboolean~5fexpressions--full--b8dedc0a-4d0b-51f4-b612-1ae854aab368","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not (not (not not not (max(9, -6, -2, 5) - min(9, -6, -2, 5) <= 4) and not not (max(1, -7, 8, -10) - min(1, -7, 8, -10) <= 10)) and (not (10 * -4 + 2 * -9 > -3 * 4) or (7 * 7 <= -6))) and ((not (True) or (6 * 5 + 4 * -7 <= -5 * -4)) and (not not not (7 * -8 > -3) and not not (2 - (10 / 9) is greater than -4)))) or (not not not (((The capital of India is New Delhi.) or (9 * -4 + 1 * 2 <= -4 * 7)) and (not (The capital of Armenia is Yerevan.) and (False))) and not not (not (not not not (6 * -5 + 9 * -4 <= -5 * 2) and not not (The capital of UAE is Dubai.)) and (not not not (The capital of Malaysia is Kuala Lumpur.) and not not (-8 - (-2 / -5) is less than or equal to 10))))\\n(B) not (not not (not not not (not (max(-5, 1, -7, -5) - min(-5, 1, -7, -5) <= 10) and (True)) and not not ((The capital of Cameroon is Yaounde.) or (False))) or not not not (not not not (not (10 * -7 is less than or equal to -2) and (-2 - (8 / -8) > 5)) and not not (not not not (The capital of Belarus is Grodno.) and not not (False)))) and (not not not ((not (-1 * 2 + -5 * 5 > 10 * 5) or (max(-6, -7, 1, -6) - min(-6, -7, 1, -6) > 9)) and (not (The capital of Cameroon is Douala.) or (max(1, 8, 7, -8) - min(1, 8, 7, -8) <= 1))) and not not (not (not not not (10 * 3 > -2) and not not (The capital of Belarus is Grodno.)) or ((False) or (max(5, 7, -4, -9) - min(5, 7, -4, -9) <= 7))))\\n(C) not not not (not ((not not not (not not (not not not (not not not (((False) and (max(8, -2, -8, -9) - min(8, -2, -8, -9) <= 7)) and (The capital of Iraq is Beirut.)) and not not (5 * -2 + 3 * -5 is greater than 8 * -4)) and not not (True)) or not not not (True)) and not not (max(4, -3, 9, 8) - min(4, -3, 9, 8) is greater than 10)) or (max(-8, -3, -2, 3) - min(-8, -3, -2, 3) is greater than 6)) or (max(3, -7, 7, 1) - min(3, -7, 7, 1) <= 8)) and not not (7 * -9 + 4 * 8 > -4 * -4)\\n(D) not (not not not (not not (7 * -7 + 1 * -3 <= -1 * -7) or not not not (-7 - (1 / -5) > -2)) and not not (not (7 - (-1 / -6) is greater than 5) or (False))) and (not not not (not not (True) or not not not (-6 * -6 + -8 * 9 > -9 * -3)) and not not (not not (The capital of Jordan is Amman.) or not not not (max(-8, 4, 5, 4) - min(-8, 4, 5, 4) is greater than 1)))\\n(E) not not not (not not not (not not not (not not not ((not not not (not not not (not not (max(10, 10, 4, -1) - min(10, 10, 4, -1) is greater than 9) or not not not (max(9, -4, 10, 9) - min(9, -4, 10, 9) <= 3)) and not not (7 * -4 + -4 * -3 > 6 * 4)) and not not (The capital of Armenia is Gyumri.)) or (3 * 2 <= 3)) and not not (True)) and not not (-5 * -6 <= 6)) and not not (The capital of Gambia is Libreville.)) and not not (max(-1, 5, 6, -6) - min(-1, 5, 6, -6) <= 4)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}