{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b210dc32-10f2-576a-b5ce-8b4b1ae88c7f","task_key":"bbeh~5fweb~5fof~5flies--full--b210dc32-10f2-576a-b5ce-8b4b1ae88c7f","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Mariah says Osborn tells the truth. Owen says Maxwell lies. Millicent says Osric tells the truth. Owyn says Olaf tells the truth. Orla says Mandy tells the truth. Murphy says Maverick tells the truth. Sherrie says either exactly two of Leda, Moses and Ozzie tell the truth or none of them. Orhan says Osmond tells the truth. Elanor says either exactly one of Dallas, Miriam and Mike tells the truth, or all three of them. Leda says either all three of Sima, Olaf and Owyn lie, or two of them tell the truth. Mike says Miriam tells the truth. Oprah says Milo tells the truth. Moses says Ozzie tells the truth. Teressa says exactly one of Shenna, Madeleine and Miguel tells the truth.Sanket says Mitchell lies. Miles says Orlando tells the truth. Marcus says Micah lies. Rashida says either exactly one of Bernita, Mona and McKenzie tells the truth, or all three of them. Fletcher says Sherrie lies. Minnie says Mehran tells the truth. Orlando says Miles tells the truth. Miriam says Mike tells the truth. Alexis says only one of Delbert, Oprah and Mina lies. Ozzie says Moses tells the truth. Muhammad says Max tells the truth. Osborn says Mariah tells the truth. Sima says either exactly two of Alexis, Orlando and Miles tell the truth or none of them. Orin says Osric lies. John says Oswald tells the truth. Mona says McKenzie tells the truth. Oswald says John tells the truth. Kandi says either exactly one of Ka, Osborn and Mariah tells the truth, or all three of them. Antwan says Crista, Uri and Owen all tell the truth. Marvin says Marina lies. Bernita says Fletcher, Orhan and Orion all tell the truth. Miguel says Madeleine lies. Delbert says either all three of Alejandro, Mandy and Orla lie, or two of them tell the truth. Olaf says Owyn tells the truth. Madeleine says Miguel lies. Alejandro says only one of Raymond, Minnie and Martha lies. Orion says Osmond lies. Crista says either all three of Ryan, Muhammad and Max lie, or two of them tell the truth. Conception says either all three of Kandi, Maverick and Murphy lie, or two of them tell the truth. Ryan says exactly two of Elanor, Marvin and Marina tells the truth. Vernell lies. Maverick says Murphy tells the truth. McKenzie says Mona tells the truth. Raymond says either exactly one of Vernell, Oswald and John tells the truth, or all three of them. Mandy says Orla tells the truth. Lalit says Micah tells the truth. Shenna says only one of Rashida, Mallory and Sanket lies. Uri says Maxwell tells the truth. Mallory says Mitchell tells the truth. Ka says only one of Antwan, Millicent and Orin lies. Mina says Milo lies. Max says Muhammad tells the truth. Christie says only one of Teressa, Lalit and Marcus lies. Martha says Mehran lies. Dallas says Christie lies. Marina says Marvin lies. Do Kandi, Ryan, and Conception tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}