{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b31d059-0f52-58b1-a4ac-b55819b51866","task_key":"bbeh~5fweb~5fof~5flies--full--4b31d059-0f52-58b1-a4ac-b55819b51866","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Shenna says either exactly one of Shaunda, Miller and Mona tells the truth, or all three of them. Kandi says either all three of Fidel, Murray and Mohamed tell the truth or only one of them. Gwenn says either exactly two of Delfina, Michelle and Osian tell the truth or none of them. Helene says Bernita lies. Amberly says Phoebe, Oakley and Mercedes all tell the truth. Sherrie says either exactly two of Andree, Orin and Maverick tell the truth or none of them. Teressa says exactly one of Shenna, Nishanth and Mariana tells the truth.Tamika says either all three of Raymond, Mia and Maximilian lie, or two of them tell the truth. Melanie says Odin lies. Jerry says only one of Jim, Melinda and Mark lies. Inga says Jerry lies. Yoland says exactly one of Sima, Odin and Melanie tell the truth. Andree says exactly one of Leda, Muhammad and Michaela tells the truth.Michelle says Osian tells the truth. Mehran says Meredith lies. Christie says exactly two of Lorine, Orlando and Madison tell the truth. Odin says Melanie lies. Miriam says Moses lies. Mona says Miller tells the truth. Vernell says either all three of Christie, Disha and John tell the truth or only one of them. Mandy says Mike tells the truth. Delfina says Helene lies. Phoebe says only one of Tamika, Oda and Mehran lies. Mustafa says Mike lies. Virginia says Natasha lies. Disha says John tells the truth. Maximilian says Mia tells the truth. Maverick says Orin tells the truth. Leda says only one of Conception, Oswald and Virginia lies. Muhammad says Michaela lies. Maurice says Marvin tells the truth. Lorine says Alexis lies. Moses says Miriam lies. Marvin says Maurice tells the truth. Madison says Orlando lies. Odell says Osborn lies. Orin says Maverick tells the truth. Michaela says Muhammad lies. Orlando says Madison lies. Jim says only one of Teressa, Matteo and Ofelia lies. Melinda says Oleg tells the truth. Miller says Mona tells the truth. Maybelle says Michael tells the truth. Osian says Michelle tells the truth. Raymond says exactly one of Kandi, Miriam and Moses tell the truth. Sima says only one of Ka, Maybelle and Margaret tells the truth. Alexis says only one of Yoland, Mandy and Mustafa lies. Shalonda says exactly two of Amberly, Osborn and Odell tells the truth. Oda says Meredith tells the truth. Nishanth says Mariana lies. Bernita says Sherrie lies. Mark says Oleg lies. Oswald says Natasha tells the truth. Murray says Mohamed tells the truth. Shaunda lies. Ka says either exactly one of Shalonda, Marvin and Maurice tells the truth, or all three of them. Osborn says Odell lies. Mia says Maximilian tells the truth. Mariana says Nishanth lies. John says Disha tells the truth. Oakley says Maynard tells the truth. Conception says Vernell lies. Ofelia says Odette lies. Fidel says Inga lies. Margaret says Michael lies. Mohamed says Murray tells the truth. Mercedes says Maynard lies. Matteo says Odette tells the truth. Do Phoebe, Miller, and Gwenn tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}