{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"49fe1a73-84d4-5e72-ad42-da3b55ac4a6b","task_key":"bbeh~5fboolean~5fexpressions--full--49fe1a73-84d4-5e72-ad42-da3b55ac4a6b","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) (not not (not not not ((not not ((not ((7 * -4 > -3) or (1 * 2 + -5 * 4 > -3 * -3)) or (-8 * -3 + -5 * 5 is greater than -4 * 1)) or (3 - (-3 / 2) is greater than 4)) or not not not (False)) or (False)) and not not (The capital of Norway is Oslo.)) or not not not (-6 * -3 + -6 * -10 is greater than -3 * -1)) and (-6 - (-10 / -9) is less than or equal to 8)\\n(B) not not (not ((not not not (not not (not not ((not ((not (not not (not ((The capital of Canada is Ottawa.) and (10 * -8 + -2 * 1 <= -5 * 8)) and (The capital of Iran is Tehran.)) or not not not (The capital of Afghanistan is Kabul.)) or (False)) or (True)) and (True)) or (2 - (1 / -7) <= -3)) or not not not (False)) or not not not (5 * 7 + -10 * 3 <= -6 * -6)) and not not (The capital of Jordan is Amman.)) and (True)) and (2 - (9 / 9) <= -6)) or not not not (max(6, -8, -10, -6) - min(6, -8, -10, -6) is greater than 7)\\n(C) not (not not (not (6 * -10 + -5 * -8 <= 8 * -2) and (-1 * 6 > 3)) or not not not (not (False) or (max(-8, -7, -10, -3) - min(-8, -7, -10, -3) is greater than 6))) and (not not not (not (-7 * 2 is less than or equal to -9) or (max(7, 8, 10, -3) - min(7, 8, 10, -3) <= 7)) and not not (not not not (True) and not not (-9 * -9 + -5 * 5 is less than or equal to 6 * 4)))\\n(D) not not not (not not not (not not not (-4 * 3 + -9 * -2 is less than or equal to -7 * 8) and not not (-2 - (7 / 4) is less than or equal to 7)) and not not (not (max(2, -8, 9, 9) - min(2, -8, 9, 9) is greater than 5) and (9 * -6 + -2 * -4 > 4 * -5))) and not not (not (not (The capital of Malaysia is Putrajaya.) or (4 * -6 + -2 * -7 > 9 * 5)) and (not not (The capital of Canada is Ottawa.) or not not not (max(-2, 4, -10, 3) - min(-2, 4, -10, 3) is greater than 1)))\\n(E) (not not not (not not not (True) and not not (max(1, -4, -3, 10) - min(1, -4, -3, 10) <= 6)) and not not ((max(10, -9, -1, -10) - min(10, -9, -1, -10) is greater than 3) and (-10 - (-5 / 1) > 7))) or (not ((The capital of Germany is Munich.) and (2 * 8 + -1 * 3 is less than or equal to -10 * 4)) and (not not not (-1 * 10 + -9 * 2 is less than or equal to 10 * -6) and not not (9 * 2 > 9)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}