{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"c252de24-ed09-51bc-9b8c-d74264987098","task_key":"bbeh~5fboolean~5fexpressions--full--c252de24-ed09-51bc-9b8c-d74264987098","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not (not not not (not (False) or (-3 * 4 + 4 * -8 > 6 * 7)) and not not (not (The capital of Denmark is Copenhagen.) or (-8 - (-5 / -7) is less than or equal to 2))) or (not ((False) or (max(5, 9, -1, 3) - min(5, 9, -1, 3) > 10)) or (not not (max(2, -7, 1, -8) - min(2, -7, 1, -8) <= 6) or not not not (8 * 8 + 2 * -6 is greater than -7 * -4)))) and ((not (not (The capital of UAE is Dubai.) and (max(-6, -1, 5, 7) - min(-6, -1, 5, 7) <= 4)) or (not not (9 * -10 + -8 * -7 is greater than 10 * -5) or not not not (-1 * -9 <= -5))) or (not (not not (1 * 2 + -10 * 2 <= 1 * 2) or not not not (2 * -10 + -6 * 6 > 5 * 9)) or (not (True) and (2 * 8 > 5))))\\n(B) ((not not not (not (True) or (False)) and not not (not (4 * -2 + -5 * 10 > 3 * -5) and (max(4, -10, -5, -10) - min(4, -10, -5, -10) <= 5))) and (not (not (-6 * 5 + 3 * -4 > -4 * 7) or (The capital of Colombia is Medellin.)) and (not (The capital of Afghanistan is Kandahar.) or (The capital of Iraq is Baghdad.)))) or (not (not not (not (-8 - (-2 / -6) is less than or equal to 10) or (False)) or not not not (not (-8 * 7 + -9 * 5 <= -8 * 9) and (-9 - (2 / 7) > 7))) and (not ((3 - (-4 / 3) is less than or equal to 5) and (False)) or (not not not (6 * -2 + -2 * 5 > 3 * 3) and not not (True))))\\n(C) not (not (not (-2 * 4 + -10 * -6 is greater than 10 * 2) and (6 * -8 is less than or equal to 5)) or (not (-7 - (-5 / 3) > 6) or (The capital of Belarus is Minsk.))) or ((not not not (The capital of Turkey is Ankara.) and not not (max(-10, -1, 9, -7) - min(-10, -1, 9, -7) is greater than 7)) and (not (8 * -4 + 6 * -8 <= 5 * 2) or (-6 * -5 <= 3)))\\n(D) not not not (not (not not not (not not (not (not not not (not not (not not not (not not ((((not (not not not (-2 * -8 > -2) and not not (-1 * -6 <= 4)) or (-6 * -8 + -7 * 9 is less than or equal to 7 * -10)) and (-5 * -5 <= 7)) or (max(4, -4, -2, 2) - min(4, -4, -2, 2) <= 2)) or (-4 * 4 > 5)) or not not not (-6 * 2 + -5 * 7 <= -7 * 2)) and not not (-7 * 5 > 5)) or not not not (The capital of Belarus is Minsk.)) and not not (The capital of Canada is Toronto.)) and (4 * 1 > 8)) or not not not (10 * 1 + -10 * -10 is less than or equal to 10 * 1)) and not not (-8 * -7 + 7 * 7 is less than or equal to -7 * 9)) or (1 * 10 + -7 * -5 is greater than -1 * 5)) and not not (The capital of Turkey is Ankara.)\\n(E) not not ((not (not not not (not (not (not (max(10, -5, -8, 9) - min(10, -5, -8, 9) is greater than 7) and (1 * 3 > -8)) or (The capital of Nepal is Kathmandu.)) and (-9 - (1 / -6) is less than or equal to -7)) and not not (10 - (1 / -6) <= 9)) and (True)) or (False)) or not not not (The capital of Iraq is Baghdad.)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}