{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a0d36587-465f-5f7b-8817-d0613d35788b","task_key":"bbeh~5fboolean~5fexpressions--full--a0d36587-465f-5f7b-8817-d0613d35788b","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not (not not (not ((max(-5, -6, -3, 5) - min(-5, -6, -3, 5) <= 2) or (The capital of Afghanistan is Kandahar.)) or (not (-2 * 10 > 7) or (True))) or not not not (not (not (2 * 10 > 10) or (6 * -6 + 9 * 7 is greater than 6 * -5)) or ((True) and (-8 * 2 + 10 * 8 is greater than -1 * -4)))) or not not not (not not not (not (not (The capital of Belarus is Grodno.) and (-1 * 8 + -1 * -1 > -9 * -6)) and (not not not (9 - (-2 / -6) is greater than -9) and not not (10 * -8 > -10))) and not not (not not ((max(-5, -6, 2, 8) - min(-5, -6, 2, 8) is greater than 5) or (The capital of Cameroon is Yaounde.)) or not not not (not (6 * 10 + -9 * 4 is less than or equal to 4 * -3) and (The capital of Armenia is Yerevan.))))\\n(B) not ((not ((not (not not (not not not (not not (not not not (-10 * -1 + 3 * -1 is greater than -9 * 9) and not not (-10 * 5 + -5 * 7 <= 10 * 8)) or not not not (7 * -8 + 3 * 2 <= -7 * -2)) and not not (-6 * -5 + 2 * -8 > 8 * 6)) or not not not (-3 * -7 + -4 * -5 is greater than 9 * -3)) and (False)) or (-9 * 5 + 10 * -7 > 8 * 10)) or (max(-2, 4, -3, -5) - min(-2, 4, -3, -5) is greater than 5)) or (9 * -7 > 5)) and (The capital of Denmark is Copenhagen.)\\n(C) not (not not (not (max(9, -5, 6, -5) - min(9, -5, 6, -5) <= 5) and (-6 * -8 <= 1)) or not not not (not (7 - (-3 / -1) <= -1) and (True))) and (not not not (not (1 * 6 + 9 * 5 is less than or equal to 7 * -2) and (The capital of Colombia is Bogota.)) and not not (not not (6 - (1 / -4) is greater than -1) or not not not (6 * -2 + 6 * -3 > -4 * 3)))\\n(D) not (((not ((not not ((not not not (not (not not (not not not (not not (not not (not not (max(-7, -6, 8, 1) - min(-7, -6, 8, 1) is greater than 3) or not not not (4 * 3 <= -3)) or not not not (-1 * 10 > -7)) or not not not (9 - (5 / 1) is less than or equal to 10)) and not not (False)) or not not not (max(5, 4, 2, -5) - min(5, 4, 2, -5) is greater than 3)) and (-9 - (-8 / -2) is less than or equal to -3)) and not not (-3 * -3 <= -2)) and (True)) or not not not (False)) or (-4 * -10 + 9 * -7 is greater than -3 * 9)) or (False)) or (The capital of Germany is Berlin.)) or (The capital of Canada is Toronto.)) and (max(-2, -7, -2, -7) - min(-2, -7, -2, -7) > 8)\\n(E) not not not (not (not not not ((not not (not (not not not (not not (not (not not not (not not not (-6 - (1 / 1) > 6) and not not (The capital of Cameroon is Yaounde.)) and not not (3 * 2 + 10 * 4 is less than or equal to -9 * -4)) or (2 * 6 <= 5)) or not not not (max(-8, 1, -8, 6) - min(-8, 1, -8, 6) <= 3)) and not not (max(-2, -3, 6, -3) - min(-2, -3, 6, -3) > 10)) and (10 * 9 <= 8)) or not not not (8 * 1 <= -3)) and (3 * 1 + 3 * -7 is greater than -4 * 6)) and not not (-3 - (5 / -1) > 8)) and (1 * -2 + -4 * 9 is greater than 10 * -9)) and not not (10 * -1 + 2 * -7 <= -8 * -5)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}