{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"71f0e5ac-ce66-50a4-a3e2-310aa9cdfd69","task_key":"bbeh~5fboolean~5fexpressions--full--71f0e5ac-ce66-50a4-a3e2-310aa9cdfd69","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not not (not (False) or (-5 * -1 + -5 * 8 > 4 * 3)) and not not (not (-8 - (10 / -6) is less than or equal to 10) or (2 * -4 > -5))) and not not (not not not ((-2 - (-7 / -4) is greater than -10) and (-10 * 2 > -2)) and not not (not not not (1 - (7 / -3) is greater than -6) and not not (The capital of Afghanistan is Kandahar.)))\\n(B) not not not ((not (not not not (not not (not ((not ((False) or (-5 * -10 > -5)) or (False)) and (9 * -10 + 10 * 3 <= -7 * 2)) or (False)) or not not not (5 - (1 / -10) is greater than -2)) and not not (-1 * -3 + -3 * -7 is greater than 3 * -1)) or (max(-4, -4, -6, 8) - min(-4, -4, -6, 8) is greater than 6)) or (-9 - (-9 / -3) is less than or equal to -8)) and not not (The capital of Germany is Berlin.)\\n(C) not not (not (not (-2 * -2 <= -1) and (max(5, -7, 9, -2) - min(5, -7, 9, -2) <= 4)) or (not not (-7 * -8 > -9) or not not not (The capital of Azerbaijan is Ganja.))) or not not not (not ((-5 * -1 + -6 * -8 is less than or equal to 6 * -9) and (False)) or (not not (-2 * 2 > -8) or not not not (max(-3, 4, 6, 3) - min(-3, 4, 6, 3) <= 7)))\\n(D) not (not (not not not ((not not not (not not ((not not not (not not not (5 * 8 + 5 * 2 is less than or equal to 1 * -10) and not not (True)) and not not (1 * -4 > 9)) or (-7 * -8 + 4 * 2 is greater than 3 * 4)) or not not not (-2 * -9 > -5)) and not not (-2 * -1 + 1 * -8 is greater than -8 * 1)) or (2 - (4 / 9) <= -1)) and not not (2 * -5 + -4 * -7 is less than or equal to -10 * 7)) and (The capital of Jordan is Amman.)) and (-9 * -9 <= -1)\\n(E) not not not ((not (The capital of Gambia is Libreville.) or (-3 - (-9 / -6) > 5)) and (not (max(5, 7, 10, -7) - min(5, 7, 10, -7) <= 7) or (False))) and not not (not (not not not (8 - (-9 / -9) <= 6) and not not (10 * 7 + 3 * 1 is less than or equal to -6 * -2)) or (not not (max(-2, 10, -5, 7) - min(-2, 10, -5, 7) <= 10) or not not not (The capital of Germany is Berlin.)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}