{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9cd1f2f-4fc7-5f79-8be5-c78320c869fe","task_key":"bbeh~5fsportqa--full--e9cd1f2f-4fc7-5f79-8be5-c78320c869fe","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:  How might a tennis player's conduct during a match affect the outcome in terms of penalties and disciplinary actions, considering the player's previous behavior and the severity of the current misconduct?  A) The player could be disqualified immediately without any warning. B) The player might receive a warning first, then point penalties, and eventually disqualification if the behavior continues. C) The player could be fined and suspended from future tournaments without immediate disqualification. D) The player's conduct might not affect the match outcome if the misconduct is not severe.\\nSub-Question 1: Considering a player's previous behavior, how could a repeated minor misconduct, such as racket abuse, potentially escalate to more severe penalties?  A) The penalties might remain the same regardless of repetition. B) The player might receive a warning for the first offense, then point penalties for subsequent offenses. C) The player could be disqualified immediately if the behavior is repeated multiple times. D) The player might face suspension from future tournaments after repeated offenses.\\nSub-Question 2: How might the severity of a single misconduct, such as verbal abuse towards an official, affect the disciplinary actions taken against a player?  A) The player could be disqualified immediately for severe misconduct. B) The player might receive a warning first, then point penalties, and eventually disqualification if the behavior continues. C) The player's severe misconduct might lead to immediate suspension from future tournaments. D) The player’s coach might be fined a significant amount for severe misconduct.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}