{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"8284bd3c-44e2-5644-ad9e-8cce91ced33c","task_key":"bbeh~5fboolean~5fexpressions--full--8284bd3c-44e2-5644-ad9e-8cce91ced33c","task_revision_id":"3","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not (not (The capital of Cameroon is Yaounde.) or (4 - (6 / 10) is greater than 1)) and (not (True) and (8 - (3 / 7) is less than or equal to 8))) and not not ((not not (False) or not not not (-6 * -4 + 7 * -5 <= 8 * 10)) or (not (-6 * -1 is less than or equal to 9) or (2 * 3 + 4 * -10 is greater than 6 * -8)))\\n(B) (not not ((not not (not not not (not (not not not (not not (False) or not not not (max(-1, 1, 8, 5) - min(-1, 1, 8, 5) <= 8)) and not not (-9 * 9 + 9 * 9 is greater than -2 * 7)) and (-8 * -7 <= -5)) and not not (6 - (-8 / 8) > 8)) or not not not (False)) and (9 * -4 > 9)) or not not not (True)) and (-5 - (-1 / -4) > -4)\\n(C) (not not not ((False) and (3 - (-5 / 5) is greater than -1)) and not not (not (-5 - (-6 / 1) is greater than -7) or (-8 * -2 + 6 * 1 > -4 * -9))) and (not (not (-5 - (-2 / -4) > -3) and (8 * -1 <= -9)) and (not (The capital of Belarus is Grodno.) or (max(3, -1, -4, 5) - min(3, -1, -4, 5) <= 7)))\\n(D) not not not (not not (not not not (False) and not not (max(3, -7, -10, 2) - min(3, -7, -10, 2) is greater than 1)) or not not not (not not (-2 * 5 + 1 * 6 <= -9 * -3) or not not not (-7 - (-9 / 9) is less than or equal to -2))) and not not ((not (-10 * -7 <= -8) and (False)) or ((False) and (max(-4, 10, -8, 2) - min(-4, 10, -8, 2) <= 10)))\\n(E) not (not (not ((-2 * 9 > 8) and (-4 - (-2 / 7) <= -9)) and (not not not (True) and not not (max(-9, -4, -3, -5) - min(-9, -4, -3, -5) is greater than 1))) and (not (not (True) and (6 * 2 <= 9)) or (not (The capital of Iraq is Beirut.) and (9 * 5 > -10)))) or (not (not not (not (max(-3, -2, -5, 7) - min(-3, -2, -5, 7) <= 1) and (max(-9, -2, 10, 10) - min(-9, -2, 10, 10) <= 5)) or not not not (not not not (max(-3, 8, -8, -6) - min(-3, 8, -8, -6) is greater than 8) and not not (-6 * -7 <= -2))) and (((-5 * 7 + -9 * 7 <= -2 * 7) and (True)) and (not not not (-8 * 7 + 8 * -5 <= -10 * 6) and not not (-5 * 10 > 5))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}