{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"955230b0-1d87-5efd-888c-f27690a3a565","task_key":"bbeh~5fboolean~5fexpressions--full--955230b0-1d87-5efd-888c-f27690a3a565","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not (not ((The capital of the Nigeria is Abuja.) and (max(-5, -1, -9, -3) - min(-5, -1, -9, -3) is greater than 4)) or ((False) and (10 * 1 + -4 * -6 is greater than -10 * 1))) and (not (not (-2 - (6 / 4) is less than or equal to 3) and (-9 * -8 <= 6)) or ((-5 - (2 / 10) > 5) or (max(3, 4, -5, -9) - min(3, 4, -5, -9) is greater than 1)))\\n(B) (not (not not not (not (not not not (not (not not ((not (-5 * -4 + 8 * 10 is greater than -1 * -3) or (max(-3, 1, -8, -8) - min(-3, 1, -8, -8) <= 5)) or (9 * -2 is less than or equal to -10)) or not not not (max(-6, 7, -7, -2) - min(-6, 7, -7, -2) is greater than 9)) and (5 * 9 + 7 * 7 is less than or equal to -2 * 1)) and not not (-5 * -2 > 9)) or (The capital of Armenia is Yerevan.)) and not not (1 * 7 > -10)) and (4 - (-3 / 2) <= -2)) or (max(6, -4, -3, -1) - min(6, -4, -3, -1) <= 7)\\n(C) not (not ((False) or (False)) and ((-3 * 1 <= -5) or (-2 * -8 + -7 * -4 is less than or equal to 6 * -2))) and (not (not (max(7, 8, -3, 1) - min(7, 8, -3, 1) <= 1) or (-10 * -6 + -7 * 7 is less than or equal to -9 * 10)) and (not (The capital of Iraq is Beirut.) or (-10 - (2 / -5) is less than or equal to 10)))\\n(D) not (((True) and (The capital of Iran is Tehran.)) or ((max(-10, -5, -10, -1) - min(-10, -5, -10, -1) <= 4) and (4 * -4 + 6 * -2 > -3 * -2))) and (not not (not not not (8 * 2 > 1) and not not (-7 - (-2 / 6) <= -8)) or not not not (not (True) and (False)))\\n(E) not not not (not not (not not not (not not (not (not (not not ((not (not not not (not (not (10 - (8 / -2) is greater than 2) and (2 * -1 + -2 * -3 is greater than -2 * 10)) and (The capital of Norway is Bergen.)) and not not (max(-10, -7, 8, 3) - min(-10, -7, 8, 3) <= 8)) or (1 * 10 > 1)) and (6 * -1 is less than or equal to 9)) or not not not (4 * -5 + 7 * -4 <= 3 * -1)) and (4 * -10 + -6 * 8 <= 2 * -8)) and (The capital of Canada is Toronto.)) or not not not (The capital of Gambia is Libreville.)) and not not (False)) or not not not (-4 - (-4 / 2) <= -5)) and not not (max(4, 6, 3, -9) - min(4, 6, 3, -9) is greater than 8)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}