{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b828bd9-6cfc-5597-9197-fb0d6ac2cf02","task_key":"bbeh~5fsportqa--full--8b828bd9-6cfc-5597-9197-fb0d6ac2cf02","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   In a basketball game, if there is an interruption due to a player's injury and equipment malfunction, what are the likely procedures that should be followed according to standard basketball rules and protocols? A) The game is immediately cancelled. B) The injured player is substituted and the game continues after equipment is fixed. C) The game is paused, the injured player receives medical attention, and the equipment is replaced or fixed before the game resumes. D) The game continues without any pause, ignoring the injured player and malfunctioning equipment.   \\nSub-Question 1:   What are the typical responsibilities of the referee during a game interruption due to player injury and equipment malfunction? A) The referee should halt the game and make sure the injured player gets medical attention. B) The referee should ignore the injury and focus on the game. C) The referee should ensure the faulty equipment is replaced or repaired before game resumes. D) The referee should continue the game despite the faulty equipment.   \\nSub-Question 2:   What are the likely actions of the team coach during a game interruption due to a player injury and equipment malfunction? A) The coach should replace the injured player with a substitute from the bench. B) The coach should insist the game continues despite the injury. C) The coach should ensure the safety of his/her players first before game continuation. D) The coach should ignore the faulty equipment and focus on the game.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}