{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3d8b5bd-6ba7-5b05-996d-a709dfd80217","task_key":"bbeh~5fsportqa--full--c3d8b5bd-6ba7-5b05-996d-a709dfd80217","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and a sub-question. Each question comes with…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and a sub-question. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   Why are specific positioning and movement patterns crucial for ensuring the best view of the action during a volleyball game? A) They allow the referee to see both teams' actions clearly. B) They facilitate better strategic planning by the coach. C) They help in capturing high-quality footage for broadcasting. D) They ensure safety by preventing collisions between players. \\nSub-Question 1:  Why are rotational movement patterns essential during a volleyball game? A) They ensure each player gets a chance to serve. B) They help in confusing the opponent's team. C) They allow each player to play in all positions, enhancing their skills in low or amateur level. D) They ensure equal distribution of physical exertion among players.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question is A and C and for the sub-question is D, your final answer must be \\\"AC, D\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}