{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"759764c0-8e69-5f54-8d3f-971454ae2470","task_key":"bbeh~5fweb~5fof~5flies--full--759764c0-8e69-5f54-8d3f-971454ae2470","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Alejandro tells the truth. Kristian says exactly two of Leda, Ozzie and Orin tell the truth. Micah says Marlon tells the truth. Odin says Melvin lies. McKenzie says Marlon lies. Teressa says Shaunda tells the truth. Jim says Vernell lies. Virginia says Max lies. Odile says Mina tells the truth. Jerry says exactly one of Dallas, Octavius and Mustafa tell the truth. Christie says Ka tells the truth. Tamika says either all three of Jamey, Michaela and Ollie lie, or two of them tell the truth. Olympia says Maurice tells the truth. Megan says Mina lies. Miranda says Lalit tells the truth. Orin says Ozzie lies. Vernell says exactly two of Sal, Virginia and Max tell the truth. Mackenzie says Odessa lies. Ollie says Michaela tells the truth. Martha says Ozias tells the truth. Shaunda says only one of Elanor, Micah and McKenzie tells the truth. Madeleine says Odell tells the truth. Vina says Jim lies. Melissa says Odessa tells the truth. Jamey says Teressa, Odysseus and Odin all tell the truth. Fletcher says only one of Shenna, Ovid and Sanket tells the truth. Ophelia says Kate lies. Inga says either all three of Andree, Odell and Madeleine tell the truth or only one of them. Leda says exactly two of Alejandro, Oona and Miriam tell the truth. Ovid says Marsha tells the truth. Maurice says Olympia tells the truth. Moses says Millicent tells the truth. Miriam says Oona lies. Mustafa says Octavius lies. Octavius says Mustafa lies. Rashida says either exactly two of Tamika, Ozias and Martha tell the truth or none of them. Elanor says either exactly two of Helene, Millicent and Moses tell the truth or none of them. Max says Virginia lies. Sanket says Marsha lies. Sherrie says only one of Kristian, Octavian and Owyn lies. Lalit says Miranda tells the truth. Otis says Kate tells the truth. Oona says Miriam lies. Millicent says Moses tells the truth. Ozzie says Orin lies. Shenna says Inga, Melissa and Mackenzie all tell the truth. Ozias says Martha tells the truth. Owyn says Oakleigh lies. Andree says either all three of Christie, Lalit and Miranda tell the truth or only one of them. Dallas says only one of Sherrie, Otis and Ophelia lies. Ka says either all three of Vina, Maurice and Olympia tell the truth or only one of them. Helene says only one of Fletcher, Odile and Megan lies. Octavian says Oakleigh tells the truth. Michaela says Ollie tells the truth. Odell says Madeleine tells the truth. Odysseus says Melvin tells the truth. Sal says Jerry lies. Do Elanor, Helene, and Rashida tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}