{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bf0038a7-54e5-5774-b771-1a7ca5fb0fd3","task_key":"bbeh~5fsportqa--full--bf0038a7-54e5-5774-b771-1a7ca5fb0fd3","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   What measures can be taken to ensure the smooth progression of a basketball game and to avoid unnecessary interruptions? A) Implementing strict rules and regulations about fouls and violations. B) Regularly maintaining the court and equipment. C) Providing continuous training for the referees and players about the rules of the game. D) Limiting the number of timeouts allowed per team.   \\nSub-Question 1:   What impact would implementing strict rules and regulations about fouls and violations have on the progression of a basketball game? A) It would maintain the flow of the game by discouraging intentional fouls. B) It would increase the length of the game due to frequent stoppages for foul calls. C) It would enhance the safety of the players on the court. D) It would discourage aggressive gameplay.   \\nSub-Question 2:   How can regular maintenance of the court and equipment contribute to avoiding unnecessary interruptions during a basketball game? A) It prevents delays caused by equipment malfunction or unsafe court conditions. B) It provides a consistent playing surface, reducing the risk of injuries. C) It has no impact on the progression of the game. D) It enhances the aesthetics of the game, attracting more viewers.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}