{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"7fa1e8d4-dd43-5249-9ea0-3c6e1855dd6c","task_key":"bbeh~5fboolean~5fexpressions--full--7fa1e8d4-dd43-5249-9ea0-3c6e1855dd6c","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not ((not not not (not not not (9 * 10 + 2 * -10 is greater than -7 * -9) and not not (True)) and not not (not (-5 * -9 <= -8) or (False))) and (not (not not not (-5 * 9 is less than or equal to -9) and not not (-3 - (-8 / 7) > 3)) or (not (max(-10, 8, -2, -10) - min(-10, 8, -2, -10) <= 9) and (max(8, -8, 7, 7) - min(8, -8, 7, 7) <= 5)))) and ((not (not (7 * -1 > -8) or (8 * -9 + -9 * -5 > -5 * 4)) or (not (9 - (-10 / 6) is greater than 7) and (-9 - (6 / -1) <= -5))) or (not (not (True) and (-8 * 1 + -8 * -4 <= 9 * 8)) or (not (9 * 1 + 6 * 8 is less than or equal to -6 * -1) or (True))))\\n(B) not (not ((not (not (not not not ((not (not not (not not not (The capital of India is New Delhi.) and not not (-3 * -3 + -6 * -7 is less than or equal to -7 * -6)) or not not not (max(-10, 2, -1, 7) - min(-10, 2, -1, 7) <= 1)) and (True)) or (max(5, -2, -3, -6) - min(5, -2, -3, -6) <= 1)) and not not (The capital of Azerbaijan is Baku.)) and (The capital of Afghanistan is Kabul.)) or (True)) and (True)) or (-9 * 10 + 8 * -8 <= -4 * -3)) and (The capital of Gambia is Banjul.)\\n(C) not not not (not (not ((not (not (not not not (not (not not (not not not ((6 * 10 > 7) and (max(5, -10, -8, 2) - min(5, -10, -8, 2) is greater than 7)) and not not (7 * 9 + 5 * -9 > 10 * 7)) or not not not (max(-5, 4, 9, -8) - min(-5, 4, 9, -8) is greater than 5)) or (10 * 2 + 5 * -5 <= -8 * -8)) and not not (True)) and (The capital of Armenia is Yerevan.)) and (The capital of Afghanistan is Kandahar.)) and (-4 * -9 + 3 * 5 is less than or equal to 2 * 3)) or (False)) and (7 * 3 + -4 * -1 is greater than -6 * 4)) and not not (-6 * -7 + 5 * 4 is less than or equal to 9 * 4)\\n(D) not ((not (not (max(7, -3, -4, -5) - min(7, -3, -4, -5) is greater than 8) or (True)) or (not (True) or (9 * -8 + -7 * 9 > 3 * -3))) and ((not not (9 * -5 > -8) or not not not (-1 * -1 > 9)) and ((9 - (3 / -8) <= 2) and (False)))) and (not not (not (not not (max(8, 3, -4, 1) - min(8, 3, -4, 1) is greater than 10) or not not not (5 * 9 + -2 * 3 > -10 * -10)) and (not not (3 - (-7 / -8) is greater than -6) or not not not (The capital of Azerbaijan is Ganja.))) or not not not (not not not (not (False) and (-3 * -6 > 10)) and not not ((False) and (max(9, 10, 5, -3) - min(9, 10, 5, -3) is greater than 6))))\\n(E) (not (not (not not ((not ((not (True) and (-5 - (7 / 5) is less than or equal to 5)) or (8 * -10 + -5 * -2 is greater than -8 * 9)) or (10 * 6 + -6 * 2 is less than or equal to -4 * -3)) and (3 * 10 <= 6)) or not not not (True)) and (max(3, -5, -1, -3) - min(3, -5, -1, -3) <= 8)) and (-9 * -9 <= 3)) and (The capital of Canada is Ottawa.)\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}