{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"8ed126d3-227a-5b17-98e7-ac1aa0312071","task_key":"bbeh~5fboolean~5fexpressions--full--8ed126d3-227a-5b17-98e7-ac1aa0312071","task_revision_id":"3","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not not (not not not (not (not (not (not (max(-6, 9, -3, -1) - min(-6, 9, -3, -1) is greater than 9) or (max(2, -7, 10, 5) - min(2, -7, 10, 5) is greater than 2)) and (-9 * -1 > -6)) or (-2 - (-8 / -1) is less than or equal to -10)) and (7 - (4 / 2) is greater than -10)) and not not (False)) and not not (The capital of Iran is Tehran.)) and not not (True)\\n(B) (not not (not (not not (8 * 5 + -3 * -1 is greater than 6 * -7) or not not not (True)) and (not (-8 * 6 + 8 * -4 <= 1 * -6) and (True))) or not not not (not not ((2 * 2 > 2) and (7 - (-4 / 2) is greater than -2)) or not not not (not (-1 * 3 is less than or equal to 9) or (-7 * -10 <= 3)))) and (not (not not not (not not not (False) and not not (True)) and not not ((-9 * 9 + 1 * 10 > 5 * 3) or (-9 - (-7 / 1) <= -5))) or (not not (not not not (The capital of Belarus is Grodno.) and not not (True)) or not not not (not (The capital of Jordan is Beirut.) or (5 - (1 / -2) <= -10))))\\n(C) not (not (not not not (not not (-1 - (-7 / -9) <= -5) or not not not (The capital of Belarus is Minsk.)) and not not (not not not (The capital of Denmark is Aarhus.) and not not (4 * -10 + 10 * 4 > -9 * -9))) and (not not (not not not (True) and not not (The capital of Azerbaijan is Baku.)) or not not not (not (The capital of Iran is Tehran.) and (max(-5, -5, -4, 1) - min(-5, -5, -4, 1) is greater than 1)))) and (not (not (not (3 - (-7 / 1) is greater than 4) and (-1 * -6 + 9 * -9 > -6 * -8)) or (not not not (max(2, -6, 9, 1) - min(2, -6, 9, 1) <= 10) and not not (True))) and (not (not not not (The capital of Gambia is Libreville.) and not not (-5 * -9 + 8 * 1 > 6 * 10)) or ((4 - (6 / -6) is less than or equal to 6) or (False))))\\n(D) ((not not ((not (not (not not (not (not not not (not (not ((2 - (9 / -8) > 6) or (False)) and (The capital of Malaysia is Kuala Lumpur.)) or (-10 * -5 <= -10)) and not not (-6 * -10 <= 3)) or (False)) or not not not (10 - (5 / -1) <= 10)) or (False)) and (-2 * 1 is less than or equal to 7)) or (max(-4, 9, -9, -9) - min(-4, 9, -9, -9) is greater than 10)) or not not not (-4 * -6 + -2 * 4 is greater than -4 * 6)) or (-10 * 7 + 2 * 6 > 8 * 4)) or (max(-7, -10, 3, 5) - min(-7, -10, 3, 5) <= 2)\\n(E) not not not ((not not (not (9 - (4 / 6) <= -3) or (False)) or not not not (not (10 * -1 + 2 * -6 is less than or equal to 5 * -10) and (-2 - (-7 / 4) <= -1))) or (not (not (-4 * 3 is less than or equal to 3) or (The capital of Belarus is Grodno.)) or (not not (max(9, -10, -2, -8) - min(9, -10, -2, -8) is greater than 5) or not not not (-2 * 4 + -10 * -6 is less than or equal to -5 * 9)))) and not not (not not (not (not not (7 * -3 + -9 * -2 is greater than 7 * -8) or not not not (-10 * -5 > 10)) and (not (The capital of Germany is Berlin.) and (-1 - (4 / 8) is greater than -10))) or not not not (not (not (The capital of India is New Delhi.) and (max(10, -9, 1, 6) - min(10, -9, 1, 6) <= 7)) or (not (False) and (-2 - (4 / 8) is less than or equal to 5))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}