{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5a0ad15e-99c0-5d78-935d-bb72fd3e85c0","task_key":"bbeh~5fsportqa--full--5a0ad15e-99c0-5d78-935d-bb72fd3e85c0","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: Which of the following statements accurately reflect the specifications and requirements for a regulation-size football used in professional matches according to FIFA standards? A) The ball must have a circumference of 68-70 cm and a weight of 410-450 g. B) The ball should be spherical in shape, made of suitable material, and not pose a danger to the players. C) The ball should be inflated to an air pressure of between 0.6 and 1.1 atmospheres at sea level. D) The ball should be inflated to an air pressure of between 0.8 and 1.3 atmospheres at sea level. Sub-Question 1: According to FIFA standards, which of the following materials are suitable for making a regulation-size football used in professional matches? A) Leather B) Other materials approved by FIFA C) Plastic D) Rubber Sub-Question 2: According to FIFA standards, what are the acceptable levels of air pressure for a regulation-size football used in professional matches at sea level? A) Between 0.6 and 1.1 atmospheres B) Between 0.8 and 1.3 atmospheres  C) Between 0.5 and 1.0 atmospheres D) Between 0.7 and 1.2 atmospheres\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}