{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84edc191-42bf-5e44-9099-1fc77e6f80f7","task_key":"bbeh~5fsportqa--full--84edc191-42bf-5e44-9099-1fc77e6f80f7","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   Why is it crucial to ensure the smooth progression of a basketball game and avoid unnecessary interruptions?  A) It maintains the pace and excitement of the game. B) It allows players to maintain their momentum and focus. C) It reduces the risk of injuries. D) It ensures fair play and adherence to the rules.   \\nSub-Question 1:  Why does maintaining the pace and excitement of the game contribute to ensuring the smooth progression of a basketball game? A) It keeps the audience engaged. B) It optimizes the broadcasting and commercial aspects of the game. C) It has no impact on the game. D) It discourages players from playing well.   \\nSub-Question 2:  Why does allowing players to maintain their momentum and focus contribute to avoiding unnecessary interruptions in a basketball game? A) It ensures the players' performance is at its peak. B) It prevents players from losing concentration, which could lead to mistakes. C) It doesn't matter if players lose their focus. D) It encourages players to take unnecessary breaks.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}