{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a3b8cd7-8735-5d5a-8ecd-9fcd6fb7cb87","task_key":"bbeh~5fweb~5fof~5flies--full--4a3b8cd7-8735-5d5a-8ecd-9fcd6fb7cb87","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Maximilian says Muriel tells the truth. Martin says Oriana lies. Oriana says Martin lies. Yoland says either exactly two of Crista, Ozias and Mona tell the truth or none of them. Bernita says Tamika lies. Fidel says exactly one of Elanor, Onyx and Marty tell the truth. Madeleine says Mustafa lies. Fletcher says either exactly one of Delfina, Orson and Mario tells the truth, or all three of them. Otto says Melvin tells the truth. Kristian says Christie tells the truth. Matteo says Oran lies. Olga says Oran tells the truth. Marcus says Mallory lies. Osborn says Maverick tells the truth. Dallas says Fidel lies. Milo says Ollie lies. Ollie says Milo lies. Crista says either all three of Fletcher, Oden and Maxim lie, or two of them tell the truth. Kandi says Sima, Osborn and Molly all tell the truth. Elanor says only one of Ryan, Olympia and Mohamed tells the truth. Ozias says Mona tells the truth. Oden says Maxim tells the truth. Mona says Ozias tells the truth. Jaymie says Kandi tells the truth. Delfina says either exactly one of Jerry, Otto and Melvin tells the truth, or all three of them. Orlando says Meadow lies. Alexis says Jim lies. Mallory says Marcus lies. Alejandro says exactly one of Sal, Ollie and Milo tells the truth.Meadow says Orlando lies. Raymond says only one of Yoland, Osian and Kate tells the truth. Phoebe says Andree, Maria and Madison all tell the truth. Orin says Ocean tells the truth. Morgan says Olivia tells the truth. Vernell says Bernita tells the truth. Shaunda says exactly two of Lorine, Madeleine and Mustafa tell the truth. Ka says only one of Dallas, Olga and Matteo lies. Mary says Mindy tells the truth. Mohamed says Miles lies. Madison says Mark lies. Osian says Mitchell tells the truth. Christie says either exactly two of Shaunda, Muriel and Maximilian tell the truth or none of them. Andree says Rashida tells the truth. Onyx says Marty lies. Ocean says Orin tells the truth. Rashida says either all three of Alejandro, Olivia and Morgan tell the truth or only one of them. Maxim says Oden tells the truth. Molly says Maverick lies. Muriel says Maximilian tells the truth. Lorine says exactly one of Vernell, Mallory and Marcus tells the truth.Mario says Orson tells the truth. Olympia says Miles tells the truth. Orly says Mindy lies. Jim lies. Kate says Mitchell lies. Ryan says Kristian tells the truth. Orson says Mario tells the truth. Sima says exactly one of Phoebe, Martin and Oriana tells the truth.Mustafa says Madeleine lies. Sal says Raymond, Mary and Orly all tell the truth. Tamika says Jaymie lies. Maria says Mark tells the truth. Audrie says either exactly two of Ka, Orin and Ocean tell the truth or none of them. Marty says Onyx lies. Melvin says Otto tells the truth. Olivia says Morgan tells the truth. Jerry says exactly one of Alexis, Orlando and Meadow tell the truth. Do Jaymie, Bernita, and Audrie tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}