{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"462efdb5-07c7-5037-a82d-559680ad8702","task_key":"bbeh~5fweb~5fof~5flies--full--462efdb5-07c7-5037-a82d-559680ad8702","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Oscar says Mohamed tells the truth. Uri says Melody tells the truth. Onyx says Max lies. Micah says Mercedes lies. Audrie says only one of Shalonda, Uri and Martha tells the truth. Kandi says Inga lies. Inga says only one of Willian, Maya and Oden tells the truth. Osvaldo says Odin lies. Christie says Fletcher, Mindy and Orlando all tell the truth. Fletcher says Jim lies. Leda says exactly two of Delfina, Mandy and Osborne tells the truth. Mehran says Milton lies. Marshall says Miles tells the truth. Shalonda says Jerry, Marshall and Odysseus all tell the truth. Kristian says only one of Jaymie, Maxwell and Mona lies. Sherrie says Tamika lies. Tamika says exactly one of Lorine, Osvaldo and Odin tell the truth. Oden says Disha lies. Odysseus says Miles lies. Morgan says Oona tells the truth. Oswald says Mohamed lies. Mona says Margaret lies. Molly says Mia lies. Milton says Mehran lies. Mallory says Orla lies. Sal says exactly two of Audrie, Max and Onyx tell the truth. Maya says Disha tells the truth. Jaymie says either exactly one of Yoland, Michelle and Miriam tells the truth, or all three of them. Miguel says Meredith tells the truth. Osborne says Mandy lies. Martha says Melody lies. Conception says either exactly one of Kristian, Oona and Morgan tells the truth, or all three of them. Shaunda says exactly two of Christie, Molly and Mia tells the truth. Jim says Dallas tells the truth. Miriam says Michelle tells the truth. Mercedes says Micah lies. Meredith says Miguel tells the truth. Max says Onyx lies. Amberly says only one of Conception, Muriel and Mallory tells the truth. Orlando says Ozias lies. Oona says Morgan tells the truth. Yoland says either exactly two of Sima, Meredith and Miguel tell the truth or none of them. Mia says Molly lies. Willian says exactly two of Leda, Micah and Mercedes tells the truth. Odin says Osvaldo lies. Sima says exactly two of Sherrie, Mehran and Milton tell the truth. Dallas says exactly one of Sal, Omar and Virginia tell the truth. Delfina says Amberly, Oscar and Oswald all tell the truth. Lorine tells the truth. Michelle says Miriam tells the truth. Virginia says Omar lies. Mandy says Osborne lies. Omar says Virginia lies. Maxwell says Margaret tells the truth. Muriel says Orla tells the truth. Mindy says Ozias tells the truth. Jerry says Kandi lies. Do Inga, Miles, and Shaunda tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}