{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f29e02b9-2450-5734-8fa5-50397fa7f2df","task_key":"bbeh~5fboolean~5fexpressions--full--f29e02b9-2450-5734-8fa5-50397fa7f2df","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not ((((max(1, 1, 4, -9) - min(1, 1, 4, -9) <= 9) or (False)) and (not not not (-4 - (-4 / -7) <= -6) and not not (The capital of Nepal is Kathmandu.))) or (not (not (max(-5, 6, 9, 3) - min(-5, 6, 9, 3) <= 8) or (2 - (6 / 7) is less than or equal to 2)) and ((-7 * -4 + 10 * -9 <= 5 * 5) or (False)))) and (not (not (not not not (8 - (-2 / 5) <= -8) and not not (-1 - (-3 / 6) is greater than -2)) or (not not (-7 - (-1 / -8) > -7) or not not not (-7 * -1 + -9 * -8 is less than or equal to -5 * -8))) or (not (not not not (The capital of UAE is Dubai.) and not not (-3 * 4 + 1 * 6 is greater than -4 * 8)) and (not not (5 * 7 > 10) or not not not (max(9, 2, -10, -1) - min(9, 2, -10, -1) is greater than 4))))\\n(B) not not not (not not not (not not (not not (not (((not ((not not (not not (not (not (not not (not (True) or (max(7, -4, -2, -2) - min(7, -4, -2, -2) is greater than 5)) or not not not (True)) and (7 * 8 + -4 * 8 is greater than -3 * 7)) or (6 * 9 + 3 * 8 is less than or equal to 6 * 7)) or not not not (-8 * 10 + 9 * 8 <= 4 * 6)) or not not not (-9 * 8 + -7 * -6 <= -9 * 3)) or (False)) and (max(3, -6, -9, 2) - min(3, -6, -9, 2) is greater than 6)) or (False)) or (max(-10, -5, -3, -5) - min(-10, -5, -3, -5) is greater than 4)) and (The capital of Malaysia is Putrajaya.)) or not not not (False)) or not not not (-2 - (-4 / 5) is less than or equal to 9)) and not not (The capital of Denmark is Copenhagen.)) and not not (max(6, -8, -4, 7) - min(6, -8, -4, 7) <= 4)\\n(C) not ((not (not not (not not not (not not not (not not ((not not (not (not ((not not (not ((4 * -9 > -4) and (4 * 2 > -8)) or (-2 * 4 + -6 * -5 is greater than -10 * 6)) or not not not (max(9, -2, 7, 2) - min(9, -2, 7, 2) <= 10)) and (The capital of Iraq is Baghdad.)) and (-6 - (-9 / 8) <= -10)) or (2 * 8 + 7 * 4 is greater than 8 * 5)) or not not not (max(-4, -6, 3, 2) - min(-4, -6, 3, 2) is greater than 7)) or (True)) or not not not (8 * 1 + 7 * 3 is greater than 4 * 4)) and not not (max(2, 6, 3, 5) - min(2, 6, 3, 5) <= 10)) and not not (7 * -8 + 4 * -3 <= -7 * 3)) or not not not (max(-5, 1, 5, 10) - min(-5, 1, 5, 10) is greater than 9)) and (6 * 7 <= -2)) or (7 * 9 <= 1)) or (False)\\n(D) (not (not not (((not not (not (True) and (False)) or not not not (6 * 5 <= 3)) and (True)) and (True)) or not not not (-3 - (1 / -7) is less than or equal to 9)) and (-4 * 2 + 10 * 1 > 7 * 7)) and (-3 * 3 + -6 * -10 is less than or equal to 4 * 1)\\n(E) not (not not (not (not not not (not not (not not not (not (not not ((not (not not not (False) and not not (max(-9, 8, 7, 6) - min(-9, 8, 7, 6) is greater than 8)) or (The capital of Jordan is Amman.)) or (1 * 10 + -7 * -9 is greater than 8 * -9)) or not not not (1 - (-9 / -1) > 4)) and (True)) and not not (7 * 6 + 1 * 3 <= -6 * -10)) or not not not (True)) and not not (-9 - (-4 / 8) > -2)) and (False)) or not not not (The capital of Jordan is Beirut.)) and (False)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}