{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f16e71f-41b5-5be0-9e22-2e00478cba94","task_key":"bbeh~5fweb~5fof~5flies--full--0f16e71f-41b5-5be0-9e22-2e00478cba94","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Sal says exactly one of Tamika, Maya and Odell tells the truth.Amberly says only one of Sherrie, McKenzie and Ollie lies. Sima says Amberly lies. Ka says exactly one of Raymond, Olympia and Orlando tells the truth.Owen says Maria lies. Raymond says Sal lies. Maya says Odell lies. Oleg says Michelle tells the truth. Marlon says Oakleigh tells the truth. Orlando says Olympia lies. Jim says Ka lies. McKenzie says Mario tells the truth. Inga says exactly two of Jaymie, Moses and Melody tell the truth. Melody says Moses lies. Sherrie says exactly two of Jim, Owen and Maria tells the truth. Yoland says only one of Helene, Oliver and Melissa tells the truth. Oliver says Octavia tells the truth. Melissa says Octavia lies. Olympia says Orlando lies. Maria says Owen lies. Osian says Matthew lies. Hritik says Oakleigh lies. Odell says Maya lies. Crista says Willian tells the truth. Michelle says Oleg tells the truth. Moses says Melody lies. Misty says Matthew tells the truth. Helene lies. Tamika says either exactly one of Crista, Oleg and Michelle tells the truth, or all three of them. Vina says only one of Dallas, Misty and Osian lies. Dallas says Inga lies. Ollie says Mario lies. Willian says Yoland, Marlon and Hritik all tell the truth. Jaymie says Sima tells the truth. Do Inga, Dallas, and Vina tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}