{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99f091ca-3546-57ff-8427-97883976ce38","task_key":"bbeh~5fsportqa--full--99f091ca-3546-57ff-8427-97883976ce38","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and a sub-question. Each question comes with…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and a sub-question. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question:  Which of the following strategies would be the most effective for a table tennis sports service in order to increase its player base, considering the popularity of the sport and the demographic profile of its players?  A) Organizing high-stakes tournaments with significant cash prizes.  B) Offering personalized coaching sessions for beginners.  C) Collaborating with schools to implement table tennis in their physical education curriculum.  D) Offering discounted membership for senior citizens.\\nSub-Question 1: Which of the following is the age group which table tennis are popular in China?  A) Teenagers (13-19 years old) B) Young adults (20-29 years old) C) Middle-aged adults (30-49 years old) D) Senior citizens (50 years old and above)\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question is A and C and for the sub-question is D, your final answer must be \\\"AC, D\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}