{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7066237-80bf-5d04-a3b5-90e9c8e2cc63","task_key":"bbeh~5fsportqa--full--a7066237-80bf-5d04-a3b5-90e9c8e2cc63","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: What defensive tactics could a football team employ when facing a quarterback known for quick short passes and a running back with a high yards-after-catch average, considering the team's strength lies in their agile linebackers and a strong defensive line but weaker secondary?  A) Implement a zone defense to cover short pass routes and rely on defensive line pressure. B) Use man-to-man coverage to allow linebackers to focus on the running back. C) Apply a blitz strategy to disrupt the quarterback's rhythm. D) Focus on deep pass coverage to prevent long completions.\\nSub-Question 1: What are the tactical advantages of employing a zone defense in this scenario?  A) It allows for better run support by the linebackers. B) It can prevent the quarterback from making quick, short passes. C) It can confuse the quarterback with mixed coverage looks. D) It minimizes the impact of a weaker secondary by distributing coverage responsibilities.\\nSub-Question 2: How can a strong defensive line contribute to countering a quarterback known for quick releases?  A) By batting down passes at the line of scrimmage. B) By forcing the quarterback to throw before the receivers are ready. C) By focusing on stopping the run game exclusively. D) By dropping back into pass coverage to add depth to the secondary.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}