{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3803bd4a-f747-55da-b23d-7bc1081e3e92","task_key":"bbeh~5fsportqa--full--3803bd4a-f747-55da-b23d-7bc1081e3e92","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: During a high-stakes college football game, the head coach decides to implement a strategy that involves frequent substitutions to keep players fresh and to confuse the opposition. Given the constraints of the game, which of the following scenarios would be permissible and effective for supervising substitutions without violating any substitution rules?  A) Substituting a player who has been on the field for at least one play with a player who has been on the sidelines for the same amount of time. B) Making a double substitution after a timeout where both players have not participated in the previous play. C) Substituting two players simultaneously while the ball is in play, provided that they both come from the same positional group. D) Conducting a substitution for a player who has just participated in a play, with another who has been sitting out for a series of downs.\\nSub-Question 1: Considering the rules around substitutions in American football, which scenarios ensure that the incoming players are eligible to participate in the next play?  A) A player who has sat out for at least one play enters the field while the ball is dead and communicates with the referee. B) A player who was substituted out in the previous play re-enters the game during the same possession without any stoppage in play. C) An incoming player enters the field from the team's sideline after the previous play has ended and before the next snap. D) A player swaps positions with a teammate on the field without leaving the playing area and then participates in the next play.\\nSub-Question 2: In order to maintain strategic flexibility and adherence to substitution rules, which of the following approaches should a coach take when planning substitutions?  A) Designate specific substitution packages for different game situations and communicate these to players ahead of time. B) Allow players to decide when they need to be substituted based on their fatigue levels without any pre-established plan. C) Implement a signal system for substitutions that is understood by all players and can be executed swiftly during stoppages in play. D) Rely on random substitution patterns to prevent the opposing team from predicting the team's strategy.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}