{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c53aac9-6172-5a48-bb54-73abed29f9b0","task_key":"bbeh~5fsportqa--full--8c53aac9-6172-5a48-bb54-73abed29f9b0","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:  Why might a volleyball team prefer to use jump serves and float serves as their primary offensive strategies during a match? A) Jump serves and float serves are the easiest to execute B) Jump serves generate high speed and float serves are unpredictable in trajectory, posing challenges for the receiving team C) Jump serves and float serves require less physical exertion D) All serving strategies are equally effective, it doesn't matter which one is used   \\nSub-Question 1:  Why would a team decide to use a jump serve during a crucial point in the match? A) Jump serves are effective in surprising the opponent B) Jump serves generate high speed and are difficult to return C) Jump serves are easy to execute and require less skill D) Jump serves can be easily countered by the opposing team   \\nSub-Question 2:  Why might a float serve be an effective strategy against a team with strong receivers? A) Float serves are easy to execute and require less skill B) Float serves are predictable and easy to return C) Float serves' unpredictable trajectory can confuse even the best receivers D) Float serves generate high speed and are difficult to return\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}