{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6117b0c-af78-5a36-836f-ceb38ab321fd","task_key":"bbeh~5fsportqa--full--e6117b0c-af78-5a36-836f-ceb38ab321fd","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   How does the type of serve in volleyball influence the subsequent transition play, particularly when considering the opponents' defensive formation?  A) The type of serve has no impact on the transition play, it only depends on the opponents' defensive formation. B) The type of serve directly affects the transition play and it also depends on the opponents' defensive formation. C) The type of serve indirectly affects the transition play but does not depend on the opponents' defensive formation. D) The type of serve and the opponents' defensive formation independently influence the transition play.\\nSub-Question 1:  How does a float serve in volleyball affect the opponents' defensive formation?  A) A float serve forces the opponents to change their defensive formation. B) A float serve does not affect the opponents' defensive formation. C) A float serve can make the opponents' defensive formation unpredictable. D) A float serve can disrupt the opponents' defensive formation making it easier for the serving team to attack.\\nSub-Question 2:  How does a jump serve in volleyball influence the transition play?  A) A jump serve can lead to a quick transition play. B) A jump serve does not influence the transition play. C) A jump serve can slow down the transition play. D) A jump serve can set the rhythm for the transition play.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}