{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"41b39959-6cd3-5ef5-970b-fa607e58aa67","task_key":"bbeh~5fboolean~5fexpressions--full--41b39959-6cd3-5ef5-970b-fa607e58aa67","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not (not not not (not (not (((not ((False) or (-3 * -10 + 1 * 10 is greater than 9 * -10)) or (False)) or (4 - (9 / 2) <= -9)) and (False)) or (The capital of UAE is Dubai.)) or (False)) and not not (The capital of Germany is Berlin.)) or not not not (False)) and not not (8 * -1 + 4 * 6 is greater than -1 * 6)\\n(B) not (not (not not not (not ((not (not (not not not (not (not not not (not (((not not (-8 * 9 + -9 * 1 > -9 * 2) or not not not (-8 - (4 / 4) > 6)) and (-8 * -4 > 7)) and (False)) and (The capital of Cameroon is Yaounde.)) and not not (max(2, -1, -2, 9) - min(2, -1, -2, 9) <= 4)) or (max(4, 2, -3, -8) - min(4, 2, -3, -8) is greater than 2)) and not not (-2 * 10 is less than or equal to -9)) and (False)) and (-1 - (-9 / 3) is greater than -2)) and (max(-7, 5, -8, -1) - min(-7, 5, -8, -1) is greater than 8)) and (The capital of Jordan is Amman.)) and not not (The capital of Colombia is Bogota.)) and (-5 - (6 / -3) is less than or equal to 6)) or (8 - (3 / 6) is less than or equal to 9)\\n(C) not (not (not not not (not not (not not not (not ((not not not (not not not (not (6 * -9 is less than or equal to 1) and (The capital of Iraq is Beirut.)) and not not (The capital of Germany is Munich.)) and not not (5 - (-9 / 1) is greater than 2)) and (-10 * -5 + -1 * -2 <= -7 * -8)) or (-4 * 7 + -9 * 9 > 10 * -7)) and not not (max(1, -5, -5, -7) - min(1, -5, -5, -7) <= 6)) or not not not (4 * 7 > 5)) and not not (The capital of Malaysia is Putrajaya.)) and (-4 - (-5 / -7) is less than or equal to 9)) or (max(6, -5, -7, 3) - min(6, -5, -7, 3) <= 7)\\n(D) not not ((not (-3 - (7 / -7) is greater than -5) and (The capital of Denmark is Copenhagen.)) and (not not (2 * -2 + 1 * 3 is greater than 6 * -7) or not not not (max(-2, 10, 9, 8) - min(-2, 10, 9, 8) is greater than 1))) or not not not ((not (The capital of Cameroon is Douala.) and (-4 * -2 > 3)) and (not (1 * 1 + -10 * -8 is less than or equal to -2 * -8) and (The capital of Azerbaijan is Baku.)))\\n(E) (not not not (not not (not not (not not (not (not not (not (not (not (False) and (1 * 5 + 3 * 8 is greater than -7 * 8)) or (-6 * -8 + 10 * -10 <= -5 * -6)) or (True)) or not not not (max(7, 4, 8, 5) - min(7, 4, 8, 5) <= 2)) or (max(9, 6, 5, 7) - min(9, 6, 5, 7) <= 8)) or not not not (3 * 4 + -1 * 1 is less than or equal to -10 * 3)) or not not not (-3 * 5 > -9)) or not not not (10 - (1 / -8) <= 9)) and not not (5 * -2 is less than or equal to -10)) or (-10 - (4 / 10) > -1)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}