{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54ad6214-b677-5a9f-be24-0017db95da17","task_key":"bbeh~5fsportqa--full--54ad6214-b677-5a9f-be24-0017db95da17","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:  During a fast-paced basketball match, Team A's point guard attempted a long pass to their center who was near the opponent's basket. However, the pass was intercepted by a player from Team B, who then tried to control the ball but ended up stepping on the center line. He managed to pass the ball to a teammate in Team B's frontcourt before both his feet and the ball completely crossed to Team B's backcourt. Why was an Over-and-back violation not called in this situation?    A) The player from Team B never had full control of the ball. B) The player from Team B passed the ball before crossing the center line. C) The referee made a mistake in not calling an Over-and-back violation. D) The ball was not yet in Team B's frontcourt possession when the player touched the center line.   \\nSub-Question 1:  What does the basketball rule state about the player's position during a potential Over-and-back violation?    A) Both feet of the player and the ball must fully cross the mid-court line. B) Only the ball needs to cross the mid-court line. C) Only one foot of the player needs to cross the mid-court line. D) None of the player's feet need to cross the mid-court line.   \\nSub-Question 2:  What does the basketball rule state about possession during a potential Over-and-back violation?     A) The team must have full control of the ball in their frontcourt.  B) The team must touch the ball in their backcourt.  C) The team must have full control of the ball in their backcourt.  D) The team must not touch the ball at all.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}