{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b98b1fec-3e8b-5c32-90d1-24736229024b","task_key":"bbeh~5fweb~5fof~5flies--full--b98b1fec-3e8b-5c32-90d1-24736229024b","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Fletcher says either all three of Fidel, Margaret and Molly lie, or two of them tell the truth. Odell says Oleg lies. Miles says Marlon lies. Jim says exactly two of Ryan, Oden and Ozzie tells the truth. Mike says Osmond lies. Mateo says Mickey lies. Leda says Sal lies. Mona says Uri lies. Odysseus says Nicole lies. Margaret says Molly tells the truth. Crista says only one of Raymond, Miller and Mona lies. Ryan says only one of Kandi, Osric and Melody lies. Fidel says Rashida tells the truth. Michaela says Oran lies. Oden says Ozzie lies. Ozzie says Oden lies. Ollie says Ottoline lies. Delbert says exactly one of Crista, Mike and Osmond tells the truth.Ottoline says Ollie lies. Disha says Olaf lies. Osmond says Mike lies. Sal says exactly one of Tamika, Odell and Oleg tells the truth.Conception says only one of Fletcher, Orly and Oda lies. Tamika says exactly one of Helene, Oran and Michaela tell the truth. Sherrie tells the truth. Olaf says Disha lies. Kristian says Jamey tells the truth. Oda says Monica lies. Alejandro says only one of Leda, Odile and Odysseus tells the truth. Miller says Uri tells the truth. Mary says Obadiah lies. Oprah says Muriel lies. Mickey says Mateo lies. Jamey says exactly two of Conception, Obadiah and Mary tell the truth. Jaymie says either all three of Kristian, Orlando and Orion tell the truth or only one of them. Phoebe says Vina lies. Kandi says exactly one of Delbert, Olaf and Disha tell the truth. Michelle says Oona lies. Amberly says exactly one of Sherrie, Mickey and Mateo tell the truth. Delfina says exactly one of Phoebe, Ollie and Ottoline tells the truth.Muriel says Oprah lies. Oleg says Odell lies. Molly says Margaret tells the truth. Helene says exactly one of Jim, Oona and Michelle tells the truth.Orlando says Orion tells the truth. Audrie says exactly one of Delfina, Oprah and Muriel tell the truth. Oran says Michaela lies. Vina says exactly one of Amberly, Morris and Mohamed tells the truth.Odile says Nicole tells the truth. Oona says Michelle lies. Mohamed says Morris lies. Melody says Lalit lies. Orly says Monica tells the truth. Muhammad says Marlon tells the truth. Orion says Orlando tells the truth. Raymond says Audrie tells the truth. Morris says Mohamed lies. Rashida says Alejandro, Muhammad and Miles all tell the truth. Osric says Lalit tells the truth. Obadiah says Mary lies. Do Alejandro, Orion, and Jaymie tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}