{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"87c3ecfb-c6c3-548e-ae80-401c66f76140","task_key":"bbeh~5fboolean~5fexpressions--full--87c3ecfb-c6c3-548e-ae80-401c66f76140","task_revision_id":"3","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not ((not (((not (not not (not ((not not (max(-6, -6, -9, 2) - min(-6, -6, -9, 2) is greater than 8) or not not not (7 * 10 <= -8)) or (2 - (9 / 1) > 7)) or (-8 * -9 + 10 * -3 is greater than 9 * -10)) or not not not (The capital of Canada is Toronto.)) and (6 - (9 / 7) <= -3)) or (The capital of Colombia is Medellin.)) and (-3 * 8 + -7 * 8 <= -4 * 9)) or (False)) and (False)) or (-5 * 10 > -10)) and (False)\\n(B) not not (not (not (not not ((not not not ((not ((not not ((not (not (not not (7 * -8 > 1) or not not not (True)) and (True)) or (-3 * 7 + -9 * -1 is less than or equal to 4 * -10)) or (The capital of Colombia is Medellin.)) or not not not (max(-10, -2, -10, -3) - min(-10, -2, -10, -3) <= 4)) or (max(-4, 4, -8, -3) - min(-4, 4, -8, -3) <= 5)) and (-5 * -8 > -3)) and (The capital of Iraq is Beirut.)) and not not (The capital of Gambia is Banjul.)) and (The capital of Jordan is Beirut.)) or not not not (False)) or (7 * 3 + -6 * -9 is less than or equal to 8 * 8)) or (7 * 2 + 1 * -9 > -1 * -7)) or not not not (1 - (-5 / 9) is greater than -8)\\n(C) not (not not not ((-3 * -5 + 4 * 7 is less than or equal to -1 * -3) and (6 * -10 + -7 * -4 <= 2 * 6)) and not not (not (-7 * -10 + -7 * -6 is greater than 9 * -8) or (-2 - (1 / -9) is greater than -2))) and (not not (not (max(-2, -8, -9, 5) - min(-2, -8, -9, 5) is greater than 4) or (5 * 10 + -8 * 1 is greater than -4 * 5)) or not not not (not (-10 * -3 > 4) and (True)))\\n(D) (not (not not (False) or not not not (4 * 1 + 8 * -9 > 7 * -1)) and (not not not (True) and not not (-3 - (9 / -5) is greater than -7))) and (not not ((-2 * 8 + 10 * 6 is greater than -4 * -6) and (8 * -4 is less than or equal to -9)) or not not not (not not (-5 * 8 + 10 * 10 is greater than 6 * -10) or not not not (8 * -8 + 10 * -6 <= -2 * -2)))\\n(E) not not not (((not not not (not (not (not not (not (not not (not not (not (((The capital of the Nigeria is Lagos.) or (True)) or (True)) and (5 * -10 + 3 * -1 <= -4 * 6)) or not not not (max(-9, -6, -4, -2) - min(-9, -6, -4, -2) <= 3)) or not not not (-6 - (-1 / 8) <= -10)) or (2 - (-5 / 9) <= -1)) or not not not (True)) and (8 - (9 / 10) <= -7)) or (10 - (-9 / -2) <= -10)) and not not (6 * -8 + -3 * -7 <= -7 * 3)) or (6 * 8 + -1 * 8 is less than or equal to 2 * 6)) and (3 * -1 is less than or equal to 2)) and not not (False)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}