{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"e01da686-7b18-50cd-b859-32c8d3c623d4","task_key":"bbeh~5fboolean~5fexpressions--full--e01da686-7b18-50cd-b859-32c8d3c623d4","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not (not not (not not (8 * 10 <= 5) or not not not (max(6, 4, -4, -3) - min(6, 4, -4, -3) is greater than 6)) or not not not (not (False) or (2 - (8 / 5) <= -2))) or (not not (not (The capital of Denmark is Aarhus.) and (9 * -4 + 8 * -5 <= -7 * 8)) or not not not (not not not (The capital of Gambia is Banjul.) and not not (max(7, -8, -7, 3) - min(7, -8, -7, 3) is greater than 4)))) and not not (((not not not (max(4, -9, 2, -3) - min(4, -9, 2, -3) <= 6) and not not (max(-2, -6, -5, 9) - min(-2, -6, -5, 9) is greater than 8)) and (not not not (max(3, -3, -7, 4) - min(3, -3, -7, 4) is greater than 6) and not not (7 * -2 + -3 * 8 > 1 * -2))) or (not not not (not not (max(-4, -4, 5, -6) - min(-4, -4, 5, -6) is greater than 5) or not not not (The capital of Denmark is Copenhagen.)) and not not (not (6 * 5 + 9 * -1 is less than or equal to -7 * 6) or (-7 - (-8 / 8) > 3))))\\n(B) not not not (not not not (not not not (not not (not not not ((not (not not (not (not not (2 * -2 <= -10) or not not not (The capital of Nepal is Kathmandu.)) and (3 * 9 > 1)) or not not not (6 - (3 / 4) <= -1)) and (3 * 5 > 5)) and (False)) and not not (-6 - (2 / 2) is less than or equal to 4)) or not not not (max(-1, 6, 6, -6) - min(-1, 6, 6, -6) is greater than 9)) and not not (-4 * 9 + -5 * 10 <= 5 * 1)) and not not (True)) and not not (3 * -6 is less than or equal to -4)\\n(C) not (not not not ((not (not (not (not (not not (The capital of India is New Delhi.) or not not not (-10 - (3 / 7) > -1)) and (True)) or (True)) or (max(-6, -5, -3, -1) - min(-6, -5, -3, -1) > 8)) or (-1 * 5 + 6 * 9 is less than or equal to -4 * 2)) or (False)) and not not (The capital of Nepal is Kathmandu.)) or (4 - (10 / 5) is less than or equal to 8)\\n(D) not not not (not (not (not (max(3, -3, 4, 1) - min(3, -3, 4, 1) is greater than 3) or (max(-8, -4, -5, 9) - min(-8, -4, -5, 9) is greater than 5)) and (not (True) and (-9 - (6 / -10) > 2))) or ((not (The capital of Gambia is Libreville.) and (False)) or (not (False) and (True)))) and not not (not (not not not (not not not (7 * 10 + 5 * 1 is less than or equal to -6 * 5) and not not (7 - (-5 / 5) <= 6)) and not not (not (7 * -10 + -7 * -3 <= -4 * -8) or (False))) and (not (not not (False) or not not not (False)) or (not (-6 - (9 / 10) > -2) and (-4 * 4 + 3 * 4 <= -7 * -4))))\\n(E) not (not not not (not (not (The capital of Iraq is Beirut.) or (6 * -9 + -2 * -4 <= -4 * -1)) and ((The capital of Malaysia is Kuala Lumpur.) and (4 - (-10 / -5) <= -9))) and not not (not not (not (max(-2, -1, -2, 3) - min(-2, -1, -2, 3) <= 3) and (9 * -4 + -7 * 6 <= 1 * 6)) or not not not (not not (-10 - (-4 / 10) is less than or equal to -6) or not not not (-3 * -1 + 1 * -8 <= -6 * -3)))) and (not not not (not ((3 * 7 + -10 * 8 > 1 * 8) and (-7 * -6 <= 3)) and (not (7 * -9 + 3 * -2 <= -5 * -1) and (True))) and not not (not not ((The capital of Canada is Toronto.) and (-9 - (-6 / 3) > 2)) or not not not (not (-7 * -7 + -2 * -4 is greater than -7 * -4) and (-4 * -6 + 10 * -7 > -1 * 5))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}