{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e56e0d75-2fa0-5d93-a3d7-084eb7fbce5e","task_key":"bbeh~5fsportqa--full--e56e0d75-2fa0-5d93-a3d7-084eb7fbce5e","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:  Why might a volleyball team choose to implement a swing blocking scheme rather than a traditional blocking scheme during a high-stakes match against a team known for its strong outside hitters?  A) Swing blocking is easier for teams to execute. B) Swing blocking allows for better coverage of the back court. C) Swing blocking provides a more dynamic response to strong outside hitters. D) Swing blocking reduces the risk of injuries.\\nSub-Question 1: Why is swing blocking considered more dynamic in response to strong outside hitters compared to traditional blocking?  A) Swing blocking allows for more blockers to participate. B) Swing blocking involves a running start, increasing the height of the block. C) Swing blocking allows for better coverage of the middle court. D) Swing blocking reduces the time for blockers to get in position.\\nSub-Question 2: Why might swing blocking provide better coverage of the back court in a match against a team with strong outside hitters?  A) Swing blocking allows for more players to be in the back court. B) Swing blocking causes the ball to be deflected to the back court more often. C) Swing blocking allows the back court players to better anticipate the direction of the ball and have more reflection time. D) Swing blocking involves less player movement, conserving energy for back court defense.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}