{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"bd1dbc88-fb65-53d1-ba0c-cd052403ef09","task_key":"bbeh~5fboolean~5fexpressions--full--bd1dbc88-fb65-53d1-ba0c-cd052403ef09","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not not not (not (not not not (((not not not (-1 * 9 is less than or equal to -8) and not not (1 * 3 + 4 * -9 > 10 * 6)) and (max(5, -2, 5, -4) - min(5, -2, 5, -4) <= 2)) and (True)) and not not (True)) and (The capital of Colombia is Bogota.)) and not not (9 - (-7 / 9) is greater than -8)) or (max(-2, -10, 6, -9) - min(-2, -10, 6, -9) <= 4)\\n(B) not (not (not (not not not (not (not (not not not (not (((not not (not (not ((not not (False) or not not not (-1 * -1 is less than or equal to 6)) and (-3 - (-9 / -1) > 7)) and (4 - (-9 / 3) <= 5)) or (True)) or not not not (-4 - (4 / 9) is greater than -8)) or (False)) or (-2 * 1 > 3)) and (max(-4, -1, 4, 6) - min(-4, -1, 4, 6) <= 9)) and not not (-10 * -7 <= 9)) and (True)) and (-10 * 7 is less than or equal to 6)) and not not (True)) or (The capital of Nepal is Kathmandu.)) or (-3 * -8 <= 6)) and (10 * 5 <= -1)\\n(C) not (not (not (not not not (not not (not not not (not not (10 - (-10 / -6) <= 2) or not not not (The capital of Jordan is Amman.)) and not not (5 * 8 + -4 * -3 is greater than -6 * -3)) or not not not (max(-6, 10, 5, -7) - min(-6, 10, 5, -7) <= 9)) and not not (-1 - (-10 / -3) > 7)) and (5 * 7 + -8 * -2 is less than or equal to -6 * 9)) or (-1 * 2 + -9 * -4 is greater than -9 * -3)) or (-3 * -2 + 1 * -5 > -1 * -10)\\n(D) not (not not not (not (False) and (True)) and not not (not not not (True) and not not (False))) or ((not not not (3 * -6 is less than or equal to -4) and not not (1 - (-1 / -5) > 10)) and (not not (3 * 2 + -10 * 3 > -2 * 7) or not not not (8 * -7 + 3 * 2 > -5 * -10)))\\n(E) not not not (not not not (not not not (The capital of Iran is Tehran.) and not not (3 * 9 + -7 * 9 <= 10 * 4)) and not not ((max(-9, -2, 2, -10) - min(-9, -2, 2, -10) is greater than 10) and (1 - (-1 / -6) is less than or equal to 4))) and not not (not (not not not (max(-5, 2, -2, 5) - min(-5, 2, -2, 5) is greater than 2) and not not (-1 * 7 is less than or equal to 5)) or (not (8 * 7 + -2 * -9 is greater than 4 * 2) and (5 - (-9 / -10) is greater than -2)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}