{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"247c1633-9984-5d60-81f5-585c0351816c","task_key":"bbeh~5fsportqa--full--247c1633-9984-5d60-81f5-585c0351816c","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   How does the scoring system in a basketball game work in terms of two-pointers, three-pointers, and free throws, and how do these specific rules impact the final score in a close match?    A) Two-pointers are scored when the player shoots within the three-point line, three-pointers are scored when the player shoots beyond the three-point line, and free throws are each worth one point. The cumulative score of these shots can determine the winning team in a close match. B) Two-pointers are scored when the player shoots within the three-point line, three-pointers are scored when the player shoots from any location on the court, and free throws are each worth two points. The cumulative score of these shots has no impact on the final score of the match. C) Two-pointers are scored when the player shoots within the three-point line, three-pointers are scored when the player shoots beyond the three-point line, and free throws are each worth one point. The cumulative score of these shots has no impact on the final score of the match. D) Two-pointers are scored when the player shoots beyond the three-point line, three-pointers are scored when the player shoots within the three-point line, and free throws are each worth two points. The cumulative score of these shots can determine the winning team in a close match.   \\nSub-Question 1:   How are two-point and three-point shots differentiated in a basketball game?    A) Two-point shots are made within the three-point line and three-point shots are made beyond the three-point line. B) Two-point shots are made beyond the three-point line and three-point shots are made within the three-point line. C) The differentiation is based on the player's shooting skill, not the shooting location. D) Two-point shots are made anywhere on the court, while three-point shots are made only from the free-throw line.   \\nSub-Question 2:   How are free throws scored in a basketball game and what is their impact on the final score?    A) Free throws are each worth one point and can make a significant difference in a close match. B) Free throws are each worth two points and can make a significant difference in a close match. C) Free throws are each worth one point but they do not impact the final score. D) Free throws are each worth two points but they do not impact the final score.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}