{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a3c78e85-5864-5079-a87f-821cbead6d7b","task_key":"bbeh~5fboolean~5fexpressions--full--a3c78e85-5864-5079-a87f-821cbead6d7b","task_revision_id":"4","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) (not not (((not (not not (not (not not (not (max(-5, -8, -3, 10) - min(-5, -8, -3, 10) <= 9) or (The capital of Jordan is Beirut.)) or not not not (6 * -9 is less than or equal to 5)) or (The capital of Gambia is Libreville.)) or not not not (-3 * -6 > 3)) or (-7 * -1 + 9 * -6 > 2 * -7)) and (max(-6, 6, -8, -6) - min(-6, 6, -8, -6) <= 3)) and (True)) or not not not (True)) or (False)\\n(B) not not not (not (not (2 - (2 / 10) is less than or equal to 7) and (6 - (-9 / 5) > 9)) and ((False) and (max(-1, -7, 1, -10) - min(-1, -7, 1, -10) <= 4))) and not not (not not (not not not (max(3, -7, 9, -10) - min(3, -7, 9, -10) <= 3) and not not (The capital of Iraq is Baghdad.)) or not not not (not not (10 * 9 > 9) or not not not (-4 * 6 + -2 * -2 > -1 * 8)))\\n(C) not (not ((not (not not not (not not (not (not not not (The capital of Norway is Bergen.) and not not (-1 * -2 + 5 * -1 is greater than -1 * 4)) and (2 * -6 > -8)) or not not not (The capital of Malaysia is Kuala Lumpur.)) and not not (-6 * 9 is less than or equal to 4)) and (-5 * -8 + 2 * -4 is less than or equal to -2 * 1)) and (max(4, -1, 10, -4) - min(4, -1, 10, -4) <= 10)) and (False)) and (-5 * 8 + 9 * -1 > -4 * 9)\\n(D) not not not ((not not not (6 * -8 + 5 * 3 <= 4 * 2) and not not (10 - (9 / -5) <= -6)) or (not (-7 * -1 <= -6) or (-3 * -10 > 3))) and not not ((not not not (The capital of Denmark is Copenhagen.) and not not (max(4, 5, -7, -2) - min(4, 5, -7, -2) is greater than 5)) and (not not not (7 - (3 / 1) is greater than -4) and not not (The capital of Nepal is Pokhara.)))\\n(E) not (not not (not (2 - (9 / 3) <= -2) and (False)) or not not not ((5 * -7 > 8) or (False))) or (not not (not (4 - (9 / -2) is greater than -5) and (-6 * -2 + 6 * 10 is less than or equal to -2 * 3)) or not not not (not (-6 - (3 / -10) > 5) or (-4 - (8 / -8) is less than or equal to 3)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}