{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"08a9b4f9-ae03-5e30-9e52-959348de0da5","task_key":"bbeh~5fweb~5fof~5flies--full--08a9b4f9-ae03-5e30-9e52-959348de0da5","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Osian says Ottoline tells the truth. Jamey says Alejandro lies. Oona says Virginia lies. Maverick says Mason tells the truth. Miriam says Maynard tells the truth. Sima says exactly one of Ryan, Morgan and Melinda tells the truth.Micah says Maynard lies. Oakley says Oprah lies. Inga says only one of Sima, Ovid and Marvin lies. Ryan lies. Amberly says only one of Fletcher, Odysseus and Misty lies. Lalit says Moses tells the truth. Antwan says only one of Dallas, Osian and Mustafa lies. Osric says Uri lies. Nishanth says Ozias tells the truth. Leda says exactly one of Rashida, Orly and Ollie tell the truth. Shenna says only one of Delbert, Bahar and Osric lies. Nicole says Miguel lies. Mallory says Olaf tells the truth. Ollie says Orly lies. Shaunda says only one of Helene, Margaret and Maybelle lies. Orly says Ollie lies. Dallas says only one of Yoland, Maverick and Kate tells the truth. Odysseus says Marsha tells the truth. Delfina says only one of Jamey, Miriam and Micah lies. Ozias says Nishanth tells the truth. Alejandro says either all three of Jaymie, Madeleine and Orson tell the truth or only one of them. Melinda says Morgan lies. Miles says Melody tells the truth. Christie says only one of Andree, Obadiah and Mateo lies. Margaret says Oliver tells the truth. Fletcher says Sal tells the truth. Andree says only one of Antwan, Mallory and Mark lies. Jaymie says Kristian lies. Kate says Mason lies. Mateo says Oran lies. Kristian says either exactly two of Shalonda, Melody and Miles tell the truth or none of them. Mustafa says Ottoline lies. Merlin says Moses lies. Ovid says Opal tells the truth. Maybelle says Oliver lies. Alexis says exactly one of Leda, Oona and Virginia tells the truth.Misty says Marsha lies. Matteo says Michael tells the truth. Tamika says either all three of Alexis, Michael and Matteo tell the truth or only one of them. Morgan says Melinda lies. Madeleine says Orson tells the truth. Melody says Miles tells the truth. Virginia says Oona lies. Helene says only one of Inga, Lalit and Merlin lies. John says Oprah tells the truth. Shalonda says only one of Shenna, John and Oakley tells the truth. Rashida says Shaunda tells the truth. Michael says Matteo tells the truth. Fidel says Christie lies. Marvin says Opal lies. Bahar says Uri tells the truth. Sal says Tamika lies. Mark says Olaf lies. Miguel says Nicole lies. Orson says Madeleine tells the truth. Obadiah says Oran tells the truth. Yoland says exactly two of Amberly, Nicole and Miguel tells the truth. Delbert says either exactly one of Fidel, Nishanth and Ozias tells the truth, or all three of them. Do Dallas, Amberly, and Delfina tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}