{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。\n\nBIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"noindex"},"task_id":"60aa5b68-73b9-5360-85ad-6f8404232721","task_key":"bbeh~5fboolean~5fexpressions--full--60aa5b68-73b9-5360-85ad-6f8404232721","task_revision_id":"2","upstream_id":"","short_description":"From the following five expressions, only one evaluates to true. Which one is…","config":"bbeh_boolean_expressions","split":"full","body":"{\"input\":\"From the following five expressions, only one evaluates to true. Which one is it?\\n\\n(A) not not (not (not (True) or (-9 - (-8 / 10) is less than or equal to 7)) and (not not (5 * -4 + -2 * -10 <= -4 * -1) or not not not (-6 - (3 / -4) is less than or equal to -3))) or not not not (((-4 * -6 + -3 * 9 <= -9 * -1) or (-6 * 8 is less than or equal to -8)) and (not not (True) or not not not (-6 * -7 + -9 * 8 <= 10 * -2)))\\n(B) not not (not not ((The capital of the Nigeria is Abuja.) and (The capital of Belarus is Minsk.)) or not not not (not not (10 * -4 is less than or equal to -10) or not not not (-5 - (8 / -4) > -2))) or not not not (not not not ((-1 * -8 <= -9) and (max(-9, 6, -2, -10) - min(-9, 6, -2, -10) is greater than 3)) and not not (not not (-6 * -9 + -3 * -6 is less than or equal to -10 * 2) or not not not (True)))\\n(C) not not not (not not not (not not not (False) and not not (max(-3, -6, 1, -8) - min(-3, -6, 1, -8) is greater than 3)) and not not (not (True) and (6 * 3 + 9 * 9 is less than or equal to 5 * 7))) and not not (not (not (-5 * -6 > 2) or (2 * 1 + -1 * 10 <= 3 * 3)) and (not (False) or (6 * -2 + 10 * 2 is greater than 5 * -7)))\\n(D) not (not not (not not not (not not not ((not ((not (True) and (max(-2, 2, -1, 8) - min(-2, 2, -1, 8) <= 2)) and (max(3, 1, -10, -8) - min(3, 1, -10, -8) <= 4)) and (False)) or (8 * 2 > -10)) and not not (9 * -2 + 7 * 5 is greater than -9 * 10)) and not not (4 * -5 is less than or equal to -10)) or not not not (The capital of Norway is Bergen.)) and (max(1, 4, 9, -2) - min(1, 4, 9, -2) is greater than 9)\\n(E) not (not ((not not not (True) and not not (7 * -5 > -9)) and (not not not (True) and not not (The capital of Nepal is Pokhara.))) or (not (not (False) and (max(5, -5, -6, -6) - min(5, -5, -6, -6) <= 7)) or ((2 * -4 is less than or equal to -3) or (-1 * -4 + 8 * 7 is less than or equal to 1 * -10)))) and (not (not not not (not not (The capital of the Nigeria is Abuja.) or not not not (True)) and not not (not not (6 * 4 <= 3) or not not not (The capital of Armenia is Yerevan.))) or ((not (-5 * -3 + 3 * -3 <= -5 * -2) and (max(-10, 10, 4, 4) - min(-10, 10, 4, 4) is greater than 9)) or (not not (The capital of Denmark is Copenhagen.) or not not not (-9 - (3 / 6) is less than or equal to 4))))\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}