{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d3f4d4a-8c3e-51d0-92db-92b0564efbd2","task_key":"bbeh~5fweb~5fof~5flies--full--3d3f4d4a-8c3e-51d0-92db-92b0564efbd2","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Olive says Oswin lies. Rashida says exactly one of Vina, Molly and Bahar tell the truth. Orla says Maverick lies. Mindy says Murphy lies. Antwan says either all three of Shenna, Mason and Hritik tell the truth or only one of them. Sal says only one of Crista, Owyn and Mehran lies. Maverick says Orla lies. Ozzy says Muhammad lies. Oakleigh says Mackenzie lies. Mackenzie says Oakleigh lies. Oscar says Octavius tells the truth. Orson says Octavia tells the truth. Mason says Hritik tells the truth. Octavius says Oscar tells the truth. Lalit says Martin lies. Andree says exactly one of Audrie, Osvaldo and Millie tell the truth. Misty says Olga tells the truth. Sherrie says either all three of Sal, Oscar and Octavius lie, or two of them tell the truth. Orlando says Maxwell lies. Molly says Bahar lies. Martin says Lalit lies. Osborn says Othello tells the truth. Jaymie says exactly two of Conception, Maxwell and Orlando tells the truth. Gwenn says exactly one of Delfina, Oakleigh and Mackenzie tells the truth.Bahar says Molly lies. Miriam says Melody tells the truth. Melody says Miriam tells the truth. Murray says Orly tells the truth. Oden says Madeleine tells the truth. Kristian lies. Shenna says exactly one of Alexis, Martin and Lalit tells the truth.Oona says Mustafa lies. Otis says Odysseus lies. Yoland says only one of Kristian, Marty and Osborne lies. Nicole says Marsha tells the truth. Mustafa says Oona lies. Osborne says Morris lies. Amberly says Ryan tells the truth. Olympia says Mina lies. Ovid says Othello lies. Maxim says Osian tells the truth. Conception says Sherrie, Mallory and Mindy all tell the truth. Sanket says Muhammad tells the truth. Minnie says Odysseus tells the truth. Ka says Sima lies. Willian says only one of Jim, Osborn and Ovid tells the truth. Osvaldo says Millie lies. Vernell says only one of Teressa, Sanket and Ozzy tells the truth. Marty says Morris tells the truth. Osian says Maxim tells the truth. Delfina says exactly two of Bernita, Orla and Maverick tells the truth. Vina says either all three of Fletcher, Orson and Octavia tell the truth or only one of them. Teressa says Amberly, Oden and Orhan all tell the truth. Millie says Osvaldo lies. Octavia says Orson tells the truth. Odin says Opal lies. Orly says Murray tells the truth. Audrie says only one of Tamika, Kate and Olympia lies. Bernita says Vernell lies. Maxwell says Orlando lies. Mehran says Madison lies. Sima says either exactly two of Antwan, Murray and Orly tell the truth or none of them. Jim says either exactly one of Jaymie, Melody and Miriam tells the truth, or all three of them. Orhan says Madeleine lies. Oriana says Oswin tells the truth. Christie says only one of Yoland, Minnie and Otis tells the truth. Hritik says Mason tells the truth. Marsha says Nicole tells the truth. Mallory says Murphy tells the truth. Tamika says exactly one of Rashida, Opal and Odin tells the truth.Olga says Misty tells the truth. Opal says Odin lies. Alejandro says either exactly two of Raymond, Nicole and Marsha tell the truth or none of them. Owyn says Madison tells the truth. Alexis says either exactly two of Gwenn, Misty and Olga tell the truth or none of them. Fletcher says exactly two of Ka, Mustafa and Oona tell the truth. Raymond says only one of Christie, Oriana and Olive lies. Crista says either exactly one of Andree, Osian and Maxim tells the truth, or all three of them. Kate says Mina tells the truth. Ryan says Alejandro lies. Do Delfina, Conception, and Willian tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}