{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d59f5a3f-6787-552d-ad77-d823cdadc6d1","task_key":"bbeh~5fweb~5fof~5flies--full--d59f5a3f-6787-552d-ad77-d823cdadc6d1","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Antwan says exactly one of Jamey, Olivia and Onyx tells the truth.Jerry says only one of Inga, Othello and Osvaldo tells the truth. Max says Mustafa lies. Conception says exactly two of Ka, Max and Mustafa tells the truth. Othello says Octavius tells the truth. Mark says Olaf lies. Matteo says Mitchell tells the truth. Yoland says either exactly two of Kristian, Oona and Osmond tell the truth or none of them. Jamey says either exactly two of Rashida, Mickey and Maurice tell the truth or none of them. Osvaldo says Octavius lies. Fidel says either exactly two of Andree, Mia and Oliver tell the truth or none of them. Mickey says Maurice tells the truth. Rashida tells the truth. Mavis says Oda lies. Shaunda says exactly one of Jaymie, Mark and Olaf tell the truth. Nishanth says Moses tells the truth. Onyx says Olivia lies. Kristian says either all three of Jerry, Moses and Nishanth tell the truth or only one of them. Andree says either all three of Conception, Michaela and Murray lie, or two of them tell the truth. Ozzy says Oakleigh tells the truth. Mario says Oden lies. Oden says Mario lies. Mohamed says Orion tells the truth. Inga says Fidel tells the truth. Melvin says Minerva lies. Oda says Mavis lies. Morris says Orion lies. Osmond says Oona tells the truth. Mustafa says Max lies. Jaymie says only one of Raymond, Miranda and Meredith tells the truth. Oliver says Mia tells the truth. Miriam says Oran lies. Meredith says Minnie lies. Mitchell says Matteo tells the truth. Olaf says Mark lies. Miranda says Minnie tells the truth. Alexis says Delfina lies. Shalonda says Tamika, Mohamed and Morris all tell the truth. Michaela says Murray tells the truth. Olivia says Onyx lies. Delfina says Shenna tells the truth. Murray says Michaela tells the truth. Oran says Miriam lies. Mia says Oliver tells the truth. Tamika says only one of Shaunda, Ozzy and Octavio tells the truth. Maurice says Mickey tells the truth. Ka says exactly one of Antwan, Oden and Mario tell the truth. Helene says exactly one of Yoland, Mavis and Oda tells the truth.Oona says Osmond tells the truth. Vernell says exactly one of Shalonda, Oran and Miriam tells the truth.Moses says Nishanth tells the truth. Shenna says only one of Helene, Malcolm and Melvin lies. Raymond says either exactly one of Alexis, Mitchell and Matteo tells the truth, or all three of them. Malcolm says Minerva tells the truth. Octavio says Oakleigh lies. Do Alexis, Yoland, and Vernell tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}