{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7dec5bfe-5ec4-51d1-ac26-5b8438b7bcf7","task_key":"bbeh~5fsportqa--full--7dec5bfe-5ec4-51d1-ac26-5b8438b7bcf7","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: Which type of tennis court requires specific equipment, a denser string pattern racquet and high-altitude tennis balls, to optimize play due to its unique surface and altitude conditions?  A) Grass court B) Clay court C) Hard court D) High-altitude court\\nSub-Question 1: Which type of tennis court is known for having a faster game speed and lower bounce, therefore demanding a denser string pattern racquet for better control?  A) Grass court B) Clay court C) Hard court D) High-altitude court\\nSub-Question 2: Which specific type of tennis ball is designed to counteract less air resistance and thus slower speed due to higher altitude conditions?  A) Regular duty tennis balls B) Extra duty tennis balls C) High-altitude tennis balls D) Stage 2 tennis balls\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}