{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8cbba510-13b0-5b98-b691-559c5e3194c4","task_key":"bbeh~5fspatial~5freasoning--full--8cbba510-13b0-5b98-b691-559c5e3194c4","task_revision_id":"1","upstream_id":"","short_description":"You have been given a tree structure with 165 nodes. key, poem, and wood are…","config":"bbeh_spatial_reasoning","split":"full","body":"{\"input\":\"You have been given a tree structure with 165 nodes. key, poem, and wood are siblings. constant has no children. finger has no children. mop has two children; current and constant. sadness has no grandchildren. activation has no children. turkey has no children. amplitude has no grandchildren. feather has no children. success has no grandchildren. leadership has no children. current has no children. contrast has no children. glass has no grandchildren. algorithm has 5 children: finger, paint, school, headphones, and plate. bird has 5 children: hate, inspiration, tape, store, and turkey. dryer has no children. zebra has no children. village has no grandchildren but has 3 children: fork, duck, and scooter. browser has no grandchildren. news has 3 children: yarn, norm, and blog. rabbit has no grandchildren. wood has 7 grandchildren: icon, pants, and structure (whose parent is river), motorcycle and plastic (whose parent is furniture), and algorithm and workflow (whose parent is eye). plugin has no children. bike has no grandchildren but has 3 children: variable, education, and principle. inspiration has no grandchildren. engagement has no children. proportion has no grandchildren. form has no grandchildren. laugh has no grandchildren. wrench has no grandchildren but has 2 children: challenge and wire. tower has no grandchildren. hate has no children. structure has 4 children: smile, valley, progress, and pepper. alert has 5 children: shield, leadership, laugh, microphone, and lesson. goat has 3 children: browser, muscle, and culture. eggplant has no children. norm has no grandchildren. castle has two children; gloves and helmet. firewall has 4 children: shrimp, zebra, sadness, and rabbit. sea has no grandchildren. pasta has no grandchildren. elephant has no children. icon has no grandchildren but has 3 children: header, squirrel, and chip. paint has no grandchildren. instance has no grandchildren. phone has 3 children: toe, block, and cry. helmet has no children. valley has no grandchildren. progress has no children. skirt has no children. tiger has two children; contrast and eggplant. wine has no children. cereal has no grandchildren. workflow has 3 children: coin, axe, and butter. rocket has no grandchildren. plate has no children. symmetry has 3 children: towel, glass, and router. axe has no grandchildren. pepper has no children. store has no children. ice has no grandchildren. block has no grandchildren. tape has no grandchildren. motorcycle has no grandchildren but has 5 children: style, radio, wall, house, and stem. stage has no children. culture has no grandchildren. function has no grandchildren. archive has two children; plugin and sea. doll has no children. anger has no grandchildren but has 5 children: pot, portrait, happiness, literature, and package. program has no children. plastic has no grandchildren but has 4 children: process, size, mushroom, and balloon. coin has no grandchildren. scissors has 5 children: skirt, cereal, wine, potato, and proportion. pants has no grandchildren but has 2 children: map and cave. cry has no children. stamp has no children. leaf has no children. key has 12 grandchildren: feather, anger, phone, and bird (whose parent is piano), conditioner, mop, and workshop (whose parent is bottle), and alert, village, performance, pasta, and ritual (whose parent is virtual). lesson has no grandchildren. ritual has 4 children: stamp, engagement, instance, and earrings. shield has no grandchildren. frog has no grandchildren but has 2 children: tone and volume. coconut has 4 children: leaf, tower, success, and program. toe has no children. earrings has no grandchildren. cucumber has no grandchildren but has 2 children: fly and money. router has no children. communication has 4 children: ice, amplitude, activation, and peace. peace has no children. school has no children. muscle has no grandchildren. towel has no children. smile has no children. performance has 3 children: elephant, belief, and rocket. potato has no children. gloves has no children. lizard has 5 children: form, stage, function, dryer, and doll. yarn has no grandchildren. trumpet has no children. drum has no grandchildren but has 4 children: sustainability, reflection, circuit, and battery. microphone has no children. poem has 17 grandchildren: drum, firewall, wrench, bike, and coconut (whose parent is town), news, tiger, archive, and frog (whose parent is password), goat, cucumber, communication, castle, and symmetry (whose parent is sword), and lizard, wind, and scissors (whose parent is landscape). shrimp has no children. pin has no children. wind has two children; trumpet and pin. butter has no children. belief has no grandchildren. workshop has no grandchildren but has 4 children: wave, steak, lion, and username. blog has no grandchildren. conditioner has no grandchildren but has 3 children: shine, brain, and scale. headphones has no grandchildren. What is the cousin of the balloon? Your final answer must be only the object name (e.g., laptop). If there are multiple objects, provide them as a comma-separated list in alphabetical order (e.g., laptop, mug). Your final answer must be only the object name (e.g., laptop). If you have not previously seen the object at the vertex, your final answer must be \\\"unknown\\\".\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}