{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f6e615a-fb7e-5c41-b67b-c1621b2e74d2","task_key":"bbeh~5fweb~5fof~5flies--full--7f6e615a-fb7e-5c41-b67b-c1621b2e74d2","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Christie says exactly one of Audrie, Osvaldo and Maybelle tells the truth.Meredith says Mariah tells the truth. Audrie says either exactly two of Gwenn, Minerva and Miguel tell the truth or none of them. Lorine says exactly two of Dallas, Moses and Nishanth tells the truth. Megan says Omar lies. Max says Melinda tells the truth. Meadow says Ofelia tells the truth. Crista says either all three of Helene, Meredith and Mariah tell the truth or only one of them. Leda says only one of Delbert, Osian and Octavian lies. Nicole says Oran lies. Willian says exactly two of Kristian, Mohamed and Octavio tell the truth. Antwan says either all three of Shalonda, Mustafa and Millicent lie, or two of them tell the truth. Minerva says Miguel tells the truth. Ka says only one of Kandi, Olga and Osmond tells the truth. Moses says Nishanth lies. Octavio says Mohamed lies. Conception says exactly two of Ka, Ollie and Oscar tell the truth. Oswin says Miles tells the truth. Oran says Nicole lies. Osmond says Marvin lies. Ollie says Oscar lies. Jim says exactly one of Willian, Marlon and Misty tell the truth. Orlando says Ofelia lies. Marlon says Misty lies. Marsha says Milo lies. Rashida tells the truth. Odessa says Mercedes lies. Maxim says Melinda lies. Octavian says Mandy lies. Orion says Odell tells the truth. Kristian says only one of Conception, Meadow and Orlando lies. Yoland says Sima lies. Millicent says Mustafa tells the truth. Mohamed says Octavio lies. Misty says Marlon lies. Melvin says Oden lies. Dallas says Christie, Ozzy and Ozias all tell the truth. Delbert says only one of Bernita, Max and Maxim lies. Minnie says Odell lies. Oscar says Ollie lies. Mustafa says Millicent tells the truth. Mercedes says Odessa lies. Muhammad says Hritik tells the truth. Ozias says Oprah lies. Alexis says exactly one of Yoland, Oran and Nicole tell the truth. Miguel says Minerva tells the truth. Osvaldo says Maybelle lies. Orin says Oden tells the truth. Olga says Marvin tells the truth. Helene says exactly two of Jerry, Mercedes and Odessa tell the truth. Teressa says exactly one of Crista, Megan and Omar tell the truth. Shalonda says Leda, Orin and Melvin all tell the truth. Maybelle says Osvaldo lies. Hritik says Muhammad tells the truth. Milo says Marsha lies. Kandi says Antwan, Orion and Minnie all tell the truth. Vina says Elanor lies. Sima says either exactly one of Jim, Muhammad and Hritik tells the truth, or all three of them. Mariah says Meredith tells the truth. Bernita says exactly one of Teressa, Milo and Marsha tells the truth.Gwenn says Rashida tells the truth. Omar says Megan lies. Jerry says either all three of Vina, Miles and Oswin tell the truth or only one of them. Osian says Mandy tells the truth. Nishanth says Moses lies. Ozzy says Oprah tells the truth. Elanor says Lorine tells the truth. Miles says Oswin tells the truth. Do Jim, Kristian, and Alexis tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}