{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"990afab8-5793-5ece-8428-be758001c8c4","task_key":"bbeh~5fboolean~5fexpressions--full--990afab8-5793-5ece-8428-be758001c8c4","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not (not (max(8, 8, 10, 9) - min(8, 8, 10, 9) > 6) and (6 - (1 / 5) is greater than -7)) and (not (max(-7, 5, -10, -8) - min(-7, 5, -10, -8) <= 1) and (9 * -8 is less than or equal to -10))) or (not not not (not not not (7 * 6 + 7 * 3 is less than or equal to -10 * -5) and not not (5 * 1 + -1 * 10 is greater than -1 * 7)) and not not (not not not (-1 * -4 + 1 * 1 is less than or equal to 9 * -3) and not not (-10 - (7 / 8) is less than or equal to 9)))\\n(B) not not not (not (not not (-5 * 9 + 3 * -5 > 1 * 6) or not not not (-9 * -3 + 1 * 3 <= 5 * 7)) and ((2 - (-10 / -6) is greater than -9) or (The capital of India is Mumbai.))) and not not ((not not not (The capital of Iran is Isfahan.) and not not (False)) or ((False) or (max(1, -7, -7, 4) - min(1, -7, -7, 4) <= 8)))\\n(C) not not not (not (not not (not not not (not (not (not not not (not not not (True) and not not (7 - (-7 / -5) <= -1)) and not not (3 * 7 <= -6)) or (max(-3, 8, -6, -10) - min(-3, 8, -6, -10) <= 1)) and (False)) and not not (-4 * 10 + 2 * -3 <= -3 * -6)) or not not not (3 * -6 + -7 * 7 <= 7 * -6)) and (5 - (7 / 7) is greater than -7)) and not not (-9 * 8 > 4)\\n(D) (not (not not (not ((((not ((not (not (not not (not not (not (not not (max(8, 9, -5, 1) - min(8, 9, -5, 1) <= 7) or not not not (max(-7, 4, -4, 9) - min(-7, 4, -4, 9) <= 5)) or (-3 * -8 <= -5)) or not not not (5 * 8 + -2 * -10 is less than or equal to 9 * 4)) or not not not (The capital of Nepal is Kathmandu.)) or (-5 * -2 <= 6)) or (-1 - (-9 / 9) <= -3)) and (-5 * -1 > 2)) or (The capital of Belarus is Minsk.)) and (-1 - (-2 / 10) > 2)) or (False)) and (-2 * -1 + -4 * -3 is greater than -10 * 7)) and (3 * -10 + 1 * 2 <= -2 * 3)) or not not not (3 - (-10 / 5) is greater than -7)) or (-3 - (4 / -1) > 3)) and (9 * 5 + -4 * -1 is greater than 7 * 3)\\n(E) not not not (not (not not not (max(-8, -10, -5, -2) - min(-8, -10, -5, -2) <= 6) and not not (5 * -1 is less than or equal to 2)) or ((-5 - (8 / 2) is less than or equal to -4) or (The capital of Nepal is Pokhara.))) and not not (not (not (The capital of Iran is Tehran.) or (True)) or (not (7 * 10 + -5 * -8 is less than or equal to -4 * -5) or (8 - (7 / 2) <= 4)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}