{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4131a175-5c96-5393-9888-de82c0083cd0","task_key":"bbeh~5fweb~5fof~5flies--full--4131a175-5c96-5393-9888-de82c0083cd0","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Shalonda says exactly two of Vernell, Ocean and Mustafa tell the truth. Mariana says Melinda tells the truth. Mehran says Oprah lies. Mustafa says Ocean lies. Bernita says Yoland tells the truth. Alejandro says Delbert tells the truth. Minerva says Oswald lies. Malcolm says Mariah lies. Sima says exactly one of Christie, Monica and Misty tells the truth.Inga says Sherrie lies. Gwenn says Teressa lies. Mohamed says Marina tells the truth. Yoland says Leda lies. Vernell says Antwan lies. Crista says exactly two of Alejandro, Orla and Miles tells the truth. Delbert says exactly two of Audrie, Michael and Milo tells the truth. Teressa says Ryan lies. Shenna says exactly two of Gwenn, Mariah and Malcolm tells the truth. Oprah says Mehran lies. Morris says Disha tells the truth. Monica says Misty lies. Raymond says either all three of Crista, Morris and Disha lie, or two of them tell the truth. Sherrie says only one of Phoebe, Michelle and Omar lies. Marsha says Mitchell lies. Ryan says Vina, Ozzy and Minerva all tell the truth. Vina says exactly two of Jaymie, Mehran and Oprah tell the truth. Disha says Morris tells the truth. Melinda says Mariana tells the truth. Antwan says Alexis tells the truth. Marina says Mohamed tells the truth. Christie says either exactly two of Inga, Melinda and Mariana tell the truth or none of them. Jaymie says Sima tells the truth. Orla says Miles lies. Miles says Orla lies. Alexis says exactly two of Jamey, Mitchell and Marsha tells the truth. Ocean says Mustafa lies. Jamey says either exactly one of Shenna, Mohamed and Marina tells the truth, or all three of them. Audrie says Bernita tells the truth. Ozzy says Oswald tells the truth. Phoebe lies. Milo says Michael lies. Michael says Milo lies. Mitchell says Marsha lies. Misty says Monica lies. Omar says Marshall lies. Michelle says Marshall tells the truth. Leda says Shalonda tells the truth. Mariah says Malcolm lies. Do Antwan, Alejandro, and Raymond tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}