{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7db71bd6-d4d4-5df5-a5e6-8ad8868f1fc8","task_key":"bbeh~5fzebra~5fpuzzles--full--7db71bd6-d4d4-5df5-a5e6-8ad8868f1fc8","task_revision_id":"1","upstream_id":"","short_description":"There are 8 people next to each other in a row in positions 1, 2, 3, 4, 5, 6, 7,…","config":"bbeh_zebra_puzzles","split":"full","body":"{\"input\":\"There are 8 people next to each other in a row in positions 1, 2, 3, 4, 5, 6, 7, 8 who have the following characteristics.\\nEveryone likes a different type of flower: peonies, dahlias, carnations, lilies, daffodils, roses, marigolds, orchids.\\nEveryone owns a different handbag brand: Balenciaga, Michael Kors, Tory Burch, Versace, Coach, Polo Ralph Lauren, Gucci, Dolce and Gabbana.\\nEveryone has a different profession: chef, spy, singer, engineer, architect, designer, actor, dancer.\\nEveryone has a different name: Jackson, Drew, Mohan, Barbara, Ali, Michael, Alex, Rajiv.\\nEveryone has a different favorite hobby: welding, mixology, kite flying, wood carving, stamp collecting, astronomy, baking, cooking.\\nEveryone likes a different favorite animal: ligers, eagles, dragonflies, cockroaches, starlings, dugongs, bison, tilapia.\\nEveryone plays a different musical instrument: piano, violin, cello, saxophone, harmonica, flute, guitar, oboe.\\nEveryone is a different nationality: Italian, Bangladeshi, German, Swiss, Swede, Serbian, Israeli, Irish.\\nUsing the clues provided below, answer the question at the end.\\nClue 1: The person who likes dugongs is the harmonica player.\\nClue 2: The person who owns a Polo Ralph Lauren is immediately to the left of the person who owns a Gucci.\\nClue 3: The person who likes astronomy is somewhere in between Rajiv and the person who owns a Balenciaga in that order.\\nClue 4: Jackson is immediately to the left of the person who likes tilapia.\\nClue 5: The person who likes kite flying is the person who owns a Balenciaga.\\nClue 6: The person who likes marigolds is next to the harmonica player.\\nClue 7: The harmonica player is at one of the ends.\\nClue 8: The Swiss is at one of the ends.\\nClue 9: Ali is at one of the ends.\\nClue 10: The person who likes carnations is the person who likes eagles.\\nClue 11: The Irish is next to the architect.\\nClue 12: The designer is somewhere to the left of the person who likes lilies.\\nClue 13: Rajiv is immediately to the left of the Serbian.\\nClue 14: The person who likes orchids is somewhere in between the person who likes astronomy and the violin player in that order.\\nClue 15: The Serbian is immediately to the left of the person who likes cockroaches.\\nClue 16: The person who owns a Coach is the German.\\nClue 17: The violin player is immediately to the left of the person who likes mixology.\\nClue 18: The person who likes mixology is immediately to the left of the guitar player.\\nClue 19: Mohan is the person who owns a Polo Ralph Lauren.\\nClue 20: The person who likes eagles is Drew.\\nClue 21: The person who owns a Dolce and Gabbana is the person who likes eagles.\\nClue 22: Barbara is immediately to the left of the person at the 5th position.\\nClue 23: The person who owns a Polo Ralph Lauren is the person who likes roses.\\nClue 24: The chef is at one of the ends.\\nClue 25: The person who likes bison is next to the Swiss.\\nClue 26: The person who likes stamp collecting is Jackson.\\nClue 27: The person who owns a Polo Ralph Lauren is the person who likes cooking.\\nClue 28: The person who likes astronomy is the saxophone player.\\nClue 29: The oboe player is the person at the 6th position.\\nClue 30: The person who likes stamp collecting is somewhere to the left of the cello player.\\nClue 31: The person who likes tilapia is immediately to the left of the engineer.\\nClue 32: The cello player is at one of the ends.\\nClue 33: Jackson is next to the Israeli.\\nClue 34: The person who owns a Michael Kors is at one of the ends.\\nClue 35: The person who likes bison is somewhere to the left of the dancer.\\nClue 36: The guitar player is somewhere in between the person who likes mixology and the person who likes dragonflies in that order.\\nClue 37: The person who owns a Balenciaga is somewhere to the left of the Swiss.\\nClue 38: The person who likes ligers is somewhere to the left of the person who owns a Dolce and Gabbana.\\nClue 39: The spy is immediately to the left of the person at the 6th position.\\nClue 40: The person who likes dahlias is immediately to the left of the actor.\\nClue 41: The person who likes tilapia is the person who likes baking.\\nClue 42: The person who owns a Balenciaga is somewhere in between the person who likes daffodils and the Irish in that order.\\nClue 43: The Israeli is the piano player.\\nClue 44: The person who likes lilies is at one of the ends.\\nClue 45: The person who owns a Dolce and Gabbana is somewhere to the left of Alex.\\nClue 46: The person who likes welding is immediately to the left of the singer.\\nClue 47: The piano player is immediately to the left of the person who owns a Tory Burch.\\nClue 48: The designer is somewhere in between the person who owns a Coach and the person who likes lilies in that order.\\nClue 49: Jackson is somewhere to the left of the person who likes marigolds.\\nClue 50: The architect is the Italian.\\nClue 51: The Swede is at one of the ends.\\n\\nQuestion: What position is the flute player at?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}