{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"0308d0b7-ca43-5f8e-a040-4371b30924c8","task_key":"bbeh~5fboolean~5fexpressions--full--0308d0b7-ca43-5f8e-a040-4371b30924c8","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not (not (8 - (-8 / 10) <= 1) or (-2 * 4 > -8)) or not not not (not (-5 - (4 / 5) <= -7) or (6 - (-2 / -7) > 10))) and not not ((not (-7 * 6 + -4 * 7 > -3 * 3) or (max(3, 3, 5, 3) - min(3, 3, 5, 3) > 3)) and ((4 - (-1 / -2) > 6) and (max(-5, 10, 8, 7) - min(-5, 10, 8, 7) <= 1)))\\n(B) not (not (not not ((not not not (not (not not not (not (not not (True) or not not not (9 - (-4 / 8) is greater than 4)) and (True)) and not not (False)) and (-5 * 2 > -4)) and not not (False)) and (-2 - (-4 / -9) <= -8)) or not not not (The capital of Belarus is Minsk.)) and (5 - (-4 / 8) <= 4)) and (The capital of Colombia is Medellin.)\\n(C) not (not ((not (not not (not not not (not not not (not (not not (not (not not (not not (not (-2 * 9 + -5 * -2 <= -5 * -5) and (-4 * -8 > -7)) or not not not (False)) or not not not (False)) and (max(9, 2, 4, 5) - min(9, 2, 4, 5) <= 1)) or not not not (1 - (7 / -7) > 6)) or (3 * -10 > -1)) and not not (max(-6, -9, -3, 8) - min(-6, -9, -3, 8) <= 1)) and not not (-10 * 4 > 7)) or not not not (5 * 5 + 6 * -10 <= 2 * 4)) or (10 * 8 > -8)) and (3 * 6 <= -2)) or (-3 * 1 + -1 * 1 <= 6 * 5)) and (max(7, 5, -3, 1) - min(7, 5, -3, 1) <= 4)\\n(D) (not not not (not (not not not (True) and not not (7 - (-10 / -10) is greater than -10)) or (not (max(10, 2, -2, -4) - min(10, 2, -2, -4) <= 7) and (False))) and not not (not not ((7 * 3 <= -2) or (-10 * -10 + 8 * -9 is less than or equal to -4 * -1)) or not not not (not not (max(-7, 10, -2, -2) - min(-7, 10, -2, -2) <= 3) or not not not (True)))) or (not not (not not (not (-4 * 6 + 6 * 3 > 2 * 10) and (4 * -9 + 8 * -5 > -7 * -8)) or not not not (not (1 * -9 + -8 * 5 > -8 * 3) or (2 * -5 is less than or equal to 9))) or not not not (not (not not (7 - (10 / 10) > 6) or not not not (10 * -3 is less than or equal to 10)) and ((-7 * 7 > -4) and (-4 * 6 + -9 * -7 <= -7 * -8))))\\n(E) (not not not ((not not (not not not (not (not not not (not not not (not (not not not (not (The capital of Iran is Isfahan.) and (-3 - (6 / 6) > -4)) and not not (-8 - (-4 / 6) > 6)) or (False)) and not not (-10 * 6 > 6)) and not not (6 * -10 + 5 * -7 <= 2 * -10)) or (max(-3, 4, 3, 9) - min(-3, 4, 3, 9) is greater than 1)) and not not (6 * 2 + 6 * -8 <= 9 * 7)) or not not not (max(-5, 4, -9, 3) - min(-5, 4, -9, 3) <= 2)) and (-5 * -6 > -9)) and not not (-6 * 9 + -7 * 6 > -2 * -1)) and (-8 - (3 / -9) > 7)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}