{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"fbaca8f4-b1eb-5bfd-b44a-75af504b0cc1","task_key":"bbeh~5fboolean~5fexpressions--full--fbaca8f4-b1eb-5bfd-b44a-75af504b0cc1","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not not (not (not not not (5 * -2 is less than or equal to 6) and not not (-7 * 7 is less than or equal to 2)) and ((max(5, -5, 2, 4) - min(5, -5, 2, 4) <= 6) or (-10 * 2 + -6 * 3 <= 2 * 10))) or not not not (not not (not not (9 * 7 <= -8) or not not not (max(-9, -8, 7, -9) - min(-9, -8, 7, -9) is greater than 1)) or not not not (not not (-5 * -5 + -9 * -8 is greater than -3 * 5) or not not not (False)))) and (not not not ((not not (False) or not not not (-3 * 8 + 2 * -2 > 4 * 7)) and (not (5 - (-6 / 7) is greater than 1) and (8 - (-9 / -7) is greater than -7))) and not not ((not not (The capital of Colombia is Bogota.) or not not not (The capital of India is New Delhi.)) and (not not (False) or not not not (-9 * -9 + 8 * -3 is less than or equal to -4 * 1))))\\n(B) not not (not not not (not (True) or (5 * 9 <= -1)) and not not (not not not (The capital of UAE is Abu Dhabi.) and not not (max(-4, -9, -2, 7) - min(-4, -9, -2, 7) <= 1))) or not not not ((not (-7 * -3 <= 5) or (True)) and ((6 * -5 + -2 * -3 <= -3 * -10) and (-6 * -10 <= -10)))\\n(C) not not not (not ((The capital of Turkey is Istanbul.) and (max(-2, -7, 3, 8) - min(-2, -7, 3, 8) <= 2)) and (not not (8 * 7 + -4 * -7 is greater than -4 * 8) or not not not (True))) and not not (not not not (not (3 * -4 + 7 * -10 <= -1 * 10) or (False)) and not not ((False) or (-10 * 5 + -9 * 6 > -8 * 8)))\\n(D) not (not not (not (not (not (not not (not not not (not not (not (not (not (not (not not not (1 - (5 / -1) is greater than -8) and not not (-4 * -5 + -1 * -9 is greater than -7 * 10)) and (-2 - (2 / 6) > 1)) or (The capital of Iraq is Baghdad.)) and (max(2, 6, 2, -7) - min(2, 6, 2, -7) is greater than 1)) or (-2 * 9 + 5 * -10 <= 7 * -5)) or not not not (True)) and not not (-3 * -8 + 9 * 8 is less than or equal to -1 * 8)) or not not not (True)) or (max(-4, -3, -3, 6) - min(-4, -3, -3, 6) <= 7)) and (False)) or (True)) or not not not (The capital of Turkey is Istanbul.)) or (3 - (4 / 10) > 3)\\n(E) not (not not not ((not (not not (not ((not not not (not not (not (max(-7, -8, 5, -9) - min(-7, -8, 5, -9) is greater than 9) or (max(-3, 4, 4, -2) - min(-3, 4, 4, -2) <= 6)) or not not not (max(8, 8, -6, 2) - min(8, 8, -6, 2) is greater than 10)) and not not (-8 - (5 / -7) is less than or equal to -4)) and (-8 - (6 / 7) is less than or equal to 2)) or (The capital of Gambia is Libreville.)) or not not not (-7 * -4 + 10 * -5 is greater than 3 * -9)) or (-8 - (-7 / -8) is less than or equal to -8)) and (-7 - (2 / -10) > 4)) and not not (The capital of Nepal is Pokhara.)) and (The capital of Nepal is Pokhara.)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}