{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36f717a0-0840-5bfd-b433-5608328c14e5","task_key":"bbeh~5fsportqa--full--36f717a0-0840-5bfd-b433-5608328c14e5","task_revision_id":"1","upstream_id":"","short_description":"You will be given a main question and two sub-questions. Each question comes…","config":"bbeh_sportqa","split":"full","body":"{\"input\":\"You will be given a main question and two sub-questions. Each question comes with multiple choices you can select from. For each question, select all the correct choices.\\n\\nMain Question: How does the material and manufacturing process of a table tennis ball influence its performance characteristics and suitability for different levels of play?  A) They do not influence the performance and suitability at all. B) The material determines the weight and bounce of the ball, while the manufacturing process ensures its roundness and durability, affecting its performance and suitability for different levels of play. C) The material and manufacturing process only influence the color of the ball. D) The material and manufacturing process influence the cost of the ball, but not its performance or suitability for different levels of play.\\nSub-Question 1: How does the material of a table tennis ball influence its weight and bounce?  A) The material does not affect the weight and bounce of the ball. B) Harder materials result in a heavier and less bouncy ball. C) Materials like ABS result in a light and bouncy ball. D) The material only affects the color of the ball, not its weight or bounce.\\nSub-Question 2: How does the manufacturing process of a table tennis ball ensure its roundness and durability?  A) The manufacturing process does  affect the roundness and durability of the ball. B) The balls are machine-pressed to ensure roundness. C) The balls are hand-made, so their roundness and durability depend on the skill of the craftsman. D) The manufacturing process only affects the cost of the ball, not its roundness or durability.\\n\\nFor each question, provide the answer as a concatenation of the correct choices.\\nSeparate the answers for the questions by comma.\\nFor example, if the correct choices for the main question are A and C, for the first sub-question is D and for the second sub-question are B and D, your final answer must be \\\"AC, D, BD\\\".\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}