{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"47b731cf-3a62-5925-b552-84bd83459ede","task_key":"bbeh~5fboolean~5fexpressions--full--47b731cf-3a62-5925-b552-84bd83459ede","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not not (not not (not (5 * -8 > 10) or (-8 - (-3 / -5) is less than or equal to -6)) or not not not (not not not (max(-6, -5, -8, -5) - min(-6, -5, -8, -5) <= 7) and not not (7 - (-9 / -9) <= 3))) and not not (not (not not (False) or not not not (max(4, -2, 5, -2) - min(4, -2, 5, -2) is greater than 4)) or (not (max(8, -2, -6, -8) - min(8, -2, -6, -8) <= 10) or (False)))\\n(B) not not ((not not not (-6 * -1 > 1) and not not (4 * 7 + 5 * 2 is less than or equal to 10 * -1)) or (not not not (The capital of Denmark is Copenhagen.) and not not (True))) or not not not (not not (not (The capital of Azerbaijan is Baku.) and (-3 - (8 / -4) > 10)) or not not not (not not (-4 * 5 > 5) or not not not (max(-4, -3, -7, 3) - min(-4, -3, -7, 3) is greater than 7)))\\n(C) not not not (not not ((not (-5 * -1 + -1 * 5 is less than or equal to -1 * 3) or (9 * -6 + -10 * 3 > 9 * -5)) or (not not (The capital of Cameroon is Douala.) or not not not (-4 * 3 + 2 * -6 is less than or equal to -10 * 3))) or not not not (not not (not not (The capital of Norway is Bergen.) or not not not (3 * 3 <= 1)) or not not not ((-4 * 4 + 8 * -4 <= -4 * 8) and (6 - (3 / -9) <= 3)))) and not not (not not not (not not not (not not not (-3 - (9 / -5) is greater than -6) and not not (9 - (4 / 6) <= -1)) and not not (not (8 * -5 + -5 * -6 > 8 * 8) and (True))) and not not ((not (The capital of Colombia is Bogota.) or (True)) and (not (-1 * 6 + -10 * -5 > -10 * -9) and (-9 * -9 > -9))))\\n(D) not not (not ((4 * -8 > -4) and (-6 * -5 > -8)) and (not (True) or (6 * 8 + -5 * -2 is greater than -8 * -1))) or not not not (not not (not not not (max(-6, 6, 4, -7) - min(-6, 6, 4, -7) is greater than 4) and not not (The capital of UAE is Abu Dhabi.)) or not not not (not (2 * -10 > -1) and (The capital of UAE is Dubai.)))\\n(E) not (not (not not (max(-2, 1, -8, 10) - min(-2, 1, -8, 10) is greater than 5) or not not not (4 * -10 + -9 * -9 is greater than -5 * -7)) or (not (-5 * 7 > 1) or (The capital of Belarus is Minsk.))) or ((not not (max(4, -4, 4, 8) - min(4, -4, 4, 8) is greater than 1) or not not not (False)) and ((8 - (-3 / -3) is greater than -10) and (False)))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}