{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96885b2d-d13e-5fa7-a569-4ed8655ca24c","task_key":"bbeh~5fsportqa--full--96885b2d-d13e-5fa7-a569-4ed8655ca24c","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: How does the division of responsibilities in an American Football team during a game scenario where the offense is on the third down with long yardage to go for a first down, and the head coach has decided to attempt a surprise onside kick on the next kickoff, affect the roles of the Special Teams Coordinator and the Offensive Coordinator?  A) The Special Teams Coordinator must prepare the kickoff team for a potential onside kick while the Offensive Coordinator focuses on the third-down play. B) The Special Teams Coordinator is responsible for advising the head coach on wind conditions and field position for the onside kick, while the Offensive Coordinator must design a play to gain the necessary yardage. C) The Offensive Coordinator must communicate with the quarterback about the two-play sequence, while the Special Teams Coordinator ensures the field goal unit is prepared for a quick transition. D) The Special Teams Coordinator and the Offensive Coordinator work together to misdirect the opposing team, with the Special Teams Coordinator focusing on kick return formations and the Offensive Coordinator planning a deceptive play.\\nSub-Question 1: During the described scenario, how are the responsibilities of the Special Teams Coordinator impacted when considering the surprise onside kick and the field position?  A) The Special Teams Coordinator must review player assignments for the onside kick based on the opponent's return formation. B) The Special Teams Coordinator needs to prepare the hands team for the recovery of the onside kick. C) The Special Teams Coordinator has to coordinate with the head coach regarding the strategic timing of the onside kick. D) The Special Teams Coordinator is responsible for ensuring the punting unit is ready in case of a failed third-down conversion.\\nSub-Question 2: In the same scenario, how does the Offensive Coordinator's strategy on the third down influence the team's overall game plan?  A) The Offensive Coordinator must design a play that ensures either a first down conversion or sets up a favorable position for the onside kick. B) The Offensive Coordinator has to prepare two plays in advance, one for the third-down attempt and another as a backup in case of a fourth down. C) The Offensive Coordinator should focus on time management to provide the Special Teams unit enough time to set up for the onside kick. D) The Offensive Coordinator collaborates with the quarterback to decide whether to run a play that conservatively gains yardage or to attempt a high-risk pass.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}