{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64e80d2d-4bbc-53bf-956a-fd45baba22fb","task_key":"bbeh~5fsportqa--full--64e80d2d-4bbc-53bf-956a-fd45baba22fb","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   In a basketball match, what are the possible scenarios that can occur if a team is unable to advance the ball past the mid-court line within 8 seconds, and what are the possible consequences of a Backcourt violation?    A) The opposing team is awarded the ball at the sideline, nearest the spot where the violation occurred, and the game clock is reset to 14 seconds. B) The game is paused, and the offending team is issued a warning, with the game resuming from the same position. C) The offending team is penalized by losing possession of the ball, and the opposing team receives the ball at the half-court line. D) The offending team is given a technical foul, and the opposing team is awarded one free throw.   \\nSub-Question 1:   What are the circumstances under which a team may fail to advance the ball past the mid-court line within 8 seconds?    A) The team is deliberately stalling to run down the game clock. B) The team's players are unable to break through the opposing team's defense. C) The team's player in possession of the ball is injured and unable to continue play. D) The team's player in possession of the ball loses control and is unable to recover it within the allotted time.   \\nSub-Question 2:   What are the potential scenarios that could lead to a Backcourt violation?    A) A player in possession of the ball steps on or crosses over the mid-court line, then returns to the backcourt and touches the ball. B) A player in possession of the ball passes it to a teammate who is standing in the backcourt. C) A player throws the ball off an opposing player in order to bounce it back to their own half. D) The ball is deflected by an opposing player into the backcourt, and the offensive team is first to touch it.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}