{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d273a86-250e-5f2d-946d-23e433410d0d","task_key":"bbeh~5fweb~5fof~5flies--full--4d273a86-250e-5f2d-946d-23e433410d0d","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Marsha says Oswald lies. Delfina says either all three of Shalonda, Milo and Nishanth tell the truth or only one of them. Fletcher says either exactly two of Delbert, Mitchell and Millie tell the truth or none of them. Hritik says Maria tells the truth. Mia says Osborn lies. Delbert says exactly two of Rashida, Odysseus and Octavian tell the truth. McKenzie says Monica tells the truth. Rashida says Conception tells the truth. Mickey says Omar tells the truth. Lorine says either exactly two of Sherrie, Mickey and Omar tell the truth or none of them. Conception says only one of Phoebe, Muriel and Osvaldo tells the truth. Fidel says either exactly one of Jamey, Mindy and Natasha tells the truth, or all three of them. Millicent says Osborn tells the truth. Micah says Odette tells the truth. Ofelia says Max tells the truth. Vina says either exactly one of Crista, Max and Ofelia tells the truth, or all three of them. Mark says Oliver lies. Sherrie says either all three of Amberly, Odessa and Miles tell the truth or only one of them. Crista says exactly two of Lorine, Meadow and Martha tells the truth. Octavius says Mavis tells the truth. Mavis says Octavius tells the truth. Nishanth says Milo tells the truth. Mitchell says Millie tells the truth. Oswald says Marsha lies. Kandi says Inga tells the truth. Omar says Mickey tells the truth. Inga says Vina tells the truth. Gwenn says Delfina, Ozzie and Osric all tell the truth. Andree says exactly one of Ryan, Marsha and Oswald tell the truth. Odysseus says Octavian lies. Jaymie says only one of Andree, Nicole and Mark lies. Osvaldo says Mallory lies. Amberly says Helene lies. Ryan says either all three of Alexis, Monica and McKenzie tell the truth or only one of them. Millie says Mitchell tells the truth. Milo says Nishanth tells the truth. Orson says Minerva tells the truth. Odessa says Miles tells the truth. Max says Ofelia tells the truth. Martha says Meadow lies. Maurice says Mustafa lies. Martin says Minerva lies. Ozzie says Mike tells the truth. Natasha says Mindy tells the truth. Shenna says only one of Leda, Orson and Martin lies. Shalonda says Willian lies. Muriel says Mallory tells the truth. Octavian says Odysseus lies. Phoebe says Christie, Micah and Mackenzie all tell the truth. Monica says McKenzie tells the truth. Maria says Hritik tells the truth. Miguel says Mercedes lies. Mindy says Natasha tells the truth. Nicole says Oliver tells the truth. Miles says Odessa tells the truth. Mackenzie says Odette lies. Meadow says Martha lies. Mustafa says Maurice lies. Willian says only one of Fletcher, Maxwell and Miguel lies. Leda says either exactly one of Jaymie, Maria and Hritik tells the truth, or all three of them. Alexis says only one of Kandi, Millicent and Mia tells the truth. Maxwell says Mercedes tells the truth. Jamey says either exactly one of Shenna, Mavis and Octavius tells the truth, or all three of them. Osric says Mike lies. Helene lies. Christie says exactly one of Fidel, Maurice and Mustafa tells the truth.Do Willian, Andree, and Gwenn tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}