{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bb1ba8b-2c06-5c24-8b8c-b955b0366241","task_key":"bbeh~5fsportqa--full--1bb1ba8b-2c06-5c24-8b8c-b955b0366241","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: In a volleyball match, Player A from team X deliberately harms Player B from team Y, leading to Player B's injury. The referee fails to notice the incident and the game continues. What is the most ethical and professional course of action that the coach of team X should take in this situation?  A) Ignore the incident since the referee didn't notice it. B) Substitute Player A for another player and continue the game. C) Report the incident to the referee and accept the consequences. D) Confront Player A after the match but take no further action.\\nSub-Question 1: If the coach of team X chooses to report the incident to the referee, what is the likely immediate consequence for Player A?  A) Player A will be warned by the referee but allowed to continue playing. B) Player A will be immediately ejected from the game. C) Player A will be suspended from future matches. D) Player A will be fined by the volleyball association.\\nSub-Question 2: Assuming the coach of team X ignored the incident and Player B's team Y lodged a complaint after the match, how should the volleyball association handle this situation ethically and professionally?  A) Dismiss the complaint since the referee didn't notice the incident. B) Fine team X for their coach's lack of action. C) Launch an investigation into the incident and take appropriate action based on findings. D) Suspend team X from the league until further notice.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}