{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4607b7aa-7198-54f1-8b35-7c687d5af151","task_key":"bbeh~5fsportqa--full--4607b7aa-7198-54f1-8b35-7c687d5af151","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: How might unsporting behavior in football manifest itself during a competitive match, considering both player actions and reactions from the crowd?  A) A player deliberately tripping an opponent when the referee isn't looking. B) The crowd booing a player from the opposing team each time they get the ball. C) A player passing the ball to a teammate. D) Fans cheering when their team scores a goal.\\nSub-Question 1: How does unsporting behavior from a player, such as deliberate fouling or time-wasting, affect the dynamics of the game?  A) It can lead to the player being given a yellow or red card. B) It can disrupt the flow of the game and frustrate the opposing team. C) It can result in the player scoring a goal. D) It can lead to increased support from the crowd for the player's team.\\nSub-Question 2: How can unsporting behavior from the crowd, such as booing or throwing objects onto the field, impact the players and the overall game?  A) It can distract or intimidate the players, potentially affecting their performance. B) It can lead to the game being paused or even abandoned. C) It can boost the morale of the players. D) It can lead to a penalty kick for the opposing team.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}