{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9709622-6a4c-5ac8-b73a-33a0d8496a56","task_key":"bbeh~5fsportqa--full--d9709622-6a4c-5ac8-b73a-33a0d8496a56","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: During a high-stakes college football game, the head coach of Team A decides to implement a rotation strategy to keep his key players fresh for the crucial final quarter. He substitutes his star quarterback and top linebacker after they have played a combined total of 30 minutes. The quarterback has played twice the amount of time as the linebacker when he is substituted. If the quarterback is allowed to play again after sitting out for the duration that the linebacker initially played, but the linebacker must rest for twice the length of time the quarterback initially played before returning, which quarter will both the quarterback and linebacker be eligible to play together again, assuming each quarter is 15 minutes long?  A) Second Quarter B) Third Quarter C) Fourth Quarter D) They will not be eligible to play together again in the game\\nSub-Question 1: If the quarterback played twice the amount of time as the linebacker before being substituted, and their combined playtime before substitution was 45 minutes, how many minutes did the quarterback play?  A) 20 minutes B) 30 minutes C) 75 minutes D) 100 minutes\\nSub-Question 2: Given the quarterback's initial playtime, for how many minutes must the linebacker rest before being eligible to play again?  A) 20 minutes B) 30 minutes C) 40 minutes D) 100 minutes\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}