{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"39bc5d3c-42f1-50a1-993a-dcaa10c6f25c","task_key":"bbeh~5fboolean~5fexpressions--full--39bc5d3c-42f1-50a1-993a-dcaa10c6f25c","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) (not (not (not not (max(-6, -9, 6, 5) - min(-6, -9, 6, 5) <= 8) or not not not (The capital of Turkey is Istanbul.)) or (not (The capital of Jordan is Beirut.) or (-9 * 6 + -1 * -3 > -10 * -9))) and (not (not (The capital of Norway is Bergen.) and (The capital of Malaysia is Putrajaya.)) and (not (-6 * -10 <= 9) or (True)))) and (not not ((not (max(-2, -6, -5, 3) - min(-2, -6, -5, 3) <= 8) or (6 * -2 + -2 * 3 <= 2 * 9)) and (not not (1 * -3 + 7 * 8 is greater than -7 * -1) or not not not (True))) or not not not (not not ((The capital of Malaysia is Kuala Lumpur.) and (The capital of Azerbaijan is Baku.)) or not not not (not (-9 - (-6 / -9) > 8) and (False))))\\n(B) not (((not (not (not (not not not (2 - (-7 / -4) is less than or equal to 9) and not not (max(8, 9, -9, -9) - min(8, 9, -9, -9) <= 3)) and (False)) and (max(-8, 6, -10, 7) - min(-8, 6, -10, 7) <= 10)) and (False)) and (True)) and (-10 * -2 + -10 * -6 is less than or equal to 3 * -2)) and (False)\\n(C) not not not (not (not (not not (not not not (((4 - (6 / -9) is greater than -8) and (The capital of Norway is Bergen.)) and (True)) and not not (False)) or not not not (-9 - (-9 / 3) is less than or equal to 7)) and (8 * -7 + -5 * 2 > 2 * 1)) and (10 * -6 + -5 * 10 > -10 * -1)) and not not (True)\\n(D) not (not (not not (not ((((not (not not (not not not (The capital of the Nigeria is Abuja.) and not not (4 * -2 + -1 * 1 <= -3 * 2)) or not not not (-7 * -9 + 7 * 4 is greater than -9 * -6)) or (False)) or (-10 * 6 > 3)) or (9 * -6 is less than or equal to 3)) or (The capital of Malaysia is Putrajaya.)) or (The capital of Azerbaijan is Baku.)) or not not not (False)) and (max(-8, 7, -1, 4) - min(-8, 7, -1, 4) <= 7)) and (7 * -7 + 5 * 2 > 7 * 4)\\n(E) not not not (not not ((max(-2, -1, -5, 6) - min(-2, -1, -5, 6) is greater than 7) and (False)) or not not not (not not not (-8 * -9 + 3 * -4 <= 8 * 8) and not not (The capital of Turkey is Ankara.))) and not not (not not not (not (The capital of Malaysia is Putrajaya.) or (-2 * -10 + -10 * 8 > 2 * -4)) and not not ((-9 - (-9 / -10) is less than or equal to 6) and (3 * -8 + 1 * -2 <= -4 * -9)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}