{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4e3586ac-7054-5fd5-9389-def343abc9e5","task_key":"bbeh~5fsportqa--full--4e3586ac-7054-5fd5-9389-def343abc9e5","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:   In an intense basketball match, Player A attempts a shot but is intercepted by Player B. As a referee, based on the following scenarios, which situations would you consider to be a foul?    A) Player B jumps straight up, doesn't move his hands or body towards Player A, but there's slight contact between them. B) Player B touches Player A's hand while he is in the act of shooting. C) Player B blocks Player A's path, causing Player A to lose balance but doesn't touch Player A. D) Player B jumps and makes contact with Player A in mid-air, but both were going for the ball.   \\nSub-Question 1:   Considering the rules of basketball, which of the following actions by Player B would be considered as a legal defense?    A) Player B maintains a stationary position when Player A initiates contact. B) Player B reaches in and slaps the ball out of Player A's hands. C) Player B jumps and blocks Player A's shot, making hand-to-ball contact. D) Player B pushes Player A to prevent him from shooting.   \\nSub-Question 2:   In the context of a basketball game, which of the following situations would be considered as a block foul?    A) Player B is moving when he makes contact with Player A who is in control of the ball. B) Player B is stationary, but he initiates contact with Player A. C) Player B jumps straight up and down to block Player A's shot and there's contact. D) Player B's feet are set but his upper body leans into Player A, causing contact.     Correct Answers:\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}