{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"c6b6d87d-ba71-5578-9a2e-c4966c383ffa","task_key":"bbeh~5fboolean~5fexpressions--full--c6b6d87d-ba71-5578-9a2e-c4966c383ffa","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not (not not not (not not not (not not ((not (not (7 * 3 + 5 * 5 is greater than 5 * -6) or (-7 * -6 + 5 * 5 is less than or equal to -6 * -3)) or (False)) or (-4 * 1 + 6 * 7 is less than or equal to -5 * -5)) or not not not (2 * 5 is less than or equal to 10)) and not not (True)) and not not (The capital of Azerbaijan is Ganja.)) and (5 * 6 + 8 * 8 is greater than 6 * 6)) and (8 * 10 + -4 * 5 <= 7 * 10)\\n(B) not not not (not not (((3 * 5 > 2) or (max(9, 3, 2, -1) - min(9, 3, 2, -1) <= 5)) and ((-10 * -8 + -8 * -10 is less than or equal to 7 * -10) and (False))) or not not not (not not (not not (3 * 4 + -10 * -10 is less than or equal to 1 * -10) or not not not (False)) or not not not (not (The capital of Belarus is Grodno.) and (-9 * 8 > -4)))) and not not (not not (not (not not not (max(1, -5, 7, 2) - min(1, -5, 7, 2) <= 6) and not not (True)) or (not not not (1 * 7 + 3 * -10 <= -10 * -10) and not not (10 * 3 > -8))) or not not not (not (not (9 * 5 > 9) and (True)) and (not (The capital of Armenia is Yerevan.) or (The capital of Germany is Berlin.))))\\n(C) not not (not (not (-2 * 2 is less than or equal to -4) or (max(-7, -7, 6, -6) - min(-7, -7, 6, -6) is greater than 9)) and (not (10 * 1 > 5) and (-1 * 9 is less than or equal to -2))) or not not not (not (not (-10 * -5 + -2 * 10 is greater than -10 * 1) and (-3 * -2 > -9)) and (not (-2 - (-9 / 8) <= -2) or (10 * 3 + -4 * 10 > -5 * -4)))\\n(D) (not not not (not (-6 * 9 + -8 * -9 is less than or equal to 7 * -9) or (-10 - (7 / -3) > -7)) and not not (not (10 - (-9 / -1) > 3) or (-2 - (3 / 3) is less than or equal to 8))) and (not not ((2 * 2 + -8 * 2 <= -3 * -1) or (False)) or not not not (not not not (6 * 10 <= 6) and not not (max(-2, -4, -4, -2) - min(-2, -4, -4, -2) > 9)))\\n(E) (not ((False) and (3 * 9 + 1 * -7 <= -8 * -3)) and (not not (True) or not not not (max(4, 1, -7, 4) - min(4, 1, -7, 4) is greater than 5))) and (not not not (not (-3 * 4 + -4 * -4 <= -9 * -8) and (7 - (-4 / -4) <= -1)) and not not (not not not (False) and not not (-7 - (7 / 3) is less than or equal to -7)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}