{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6ca82060-edb0-58f3-a14a-b77538e7d726","task_key":"bbeh~5fsportqa--full--6ca82060-edb0-58f3-a14a-b77538e7d726","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: How does the specific movement pattern of a volleyball player, considering their role and the ongoing play, influence the camera angle to ensure the best view of the action?  A) The movement pattern does not influence the camera angle. B) The movement pattern, combined with the player's role, can guide the camera angle to capture the most dynamic actions. C) The camera angle is determined solely by the player's role, regardless of their movement pattern. D) The camera angle is decided randomly irrespective of the player's movement pattern or role.\\nSub-Question 1: How does the role of a volleyball player affect their movement patterns during a game?  A) The role of a player doesn't affect their movement patterns. B) Defenders usually stay at the back of the court, while attackers move towards the net. C) All players follow the same movement pattern regardless of their role. D) The role of a player only affects their movement pattern during the initial serve.\\nSub-Question 2: How does the ongoing play influence the movement patterns of a volleyball player?  A) The ongoing play does not affect a player's movement pattern. B) The ongoing play affects the movement pattern as players need to react to the ball's trajectory. C) The ongoing play only influences the movement pattern of the player who served the ball. D) The ongoing play affects the movement pattern only during the final point of the match.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}