{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa84765d-1853-565f-89e6-34b72ea2a777","task_key":"bbeh~5fsportqa--full--fa84765d-1853-565f-89e6-34b72ea2a777","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   Why is the attacking player not considered offside in a scenario where he is closer to the opponent's goal line than both the ball and the second last opponent when the ball was played?  A) Because the player was in their own half of the field when the ball was played. B) Because the player received the ball directly from a goal kick. C) Because the player was not involved in active play. D) Because the player received the ball directly from a throw-in.\\nSub-Question 1:  Why is the player not considered offside if they were in their own half of the field when the ball was played?  A) Because the offside rule does not apply in the player's own half. B) Because the player had control of the ball. C) Because the player was not interfering with an opponent. D) Because the player was not gaining an advantage from being in that position.\\nSub-Question 2:  Why is the player not considered offside if they received the ball directly from a goal kick, corner kick, or throw-in?  A) Because the offside rule does not apply in these situations. B) Because the player was not interfering with an opponent. C) Because the player was not gaining an advantage from being in that position. D) Because the player had control of the ball.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}