{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"63703448-23a0-5763-917a-f6cb8ad8594a","task_key":"bbeh~5fweb~5fof~5flies--full--63703448-23a0-5763-917a-f6cb8ad8594a","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Mindy says Oona tells the truth. Sima says exactly two of Bernita, Disha and Melinda tells the truth. Oswald says Max tells the truth. Martin says Opal tells the truth. Oleg says Maxwell lies. Teressa says Jim tells the truth. Shaunda says either all three of Ryan, Muhammad and Maya tell the truth or only one of them. Maverick says Matteo tells the truth. Mary says Melvin lies. Michaela says Lalit lies. Crista says either all three of Yoland, Oswald and Max lie, or two of them tell the truth. Shalonda says either all three of Teressa, Mindy and Oona tell the truth or only one of them. Marsha says Omar lies. Audrie says Shalonda, Malcolm and Marsha all tell the truth. Andree says Jamey lies. Ozzy says Marina lies. Osian says Mike tells the truth. Fletcher says Antwan lies. Vina says either exactly two of Sherrie, Maverick and Matteo tell the truth or none of them. Orion says Mehran lies. Malcolm says Omar tells the truth. Willian tells the truth. Oliver says Lalit tells the truth. Mehran says Orion lies. Yoland says either exactly one of Ka, Miguel and Olaf tells the truth, or all three of them. Sal says exactly one of Shenna, Maxwell and Oleg tells the truth.Ottoline says Martha lies. Ka says Fletcher, Oliver and Michaela all tell the truth. Kandi says exactly two of Amberly, Ottoline and Martha tells the truth. Miguel says Olaf tells the truth. John says Maybelle lies. Melvin says Mary lies. Martha says Ottoline lies. Raymond says exactly two of Tamika, McKenzie and Ocean tell the truth. Sherrie says Andree tells the truth. Nishanth says Murray tells the truth. McKenzie says Ocean lies. Jamey says exactly two of Gwenn, Marina and Ozzy tells the truth. Matteo says Maverick tells the truth. Shenna says exactly two of Alejandro, Owyn and Miller tells the truth. Muhammad says Maya tells the truth. Jerry says Kandi tells the truth. Jim says exactly two of Crista, Maybelle and John tell the truth. Murray says Nishanth tells the truth. Tamika says exactly one of Delfina, Melvin and Mary tells the truth.Delfina says exactly two of Jerry, Mehran and Orion tells the truth. Owyn says Miller lies. Oona says Mindy tells the truth. Disha says Melinda lies. Antwan says either exactly two of Raymond, Opal and Martin tell the truth or none of them. Amberly says Sal lies. Miller says Owyn lies. Maya says Muhammad tells the truth. Olaf says Miguel tells the truth. Alejandro says Sima tells the truth. Uri says Mike lies. Melinda says Disha lies. Max says Oswald tells the truth. Maxwell says Oleg lies. Gwenn says either exactly one of Audrie, Nishanth and Murray tells the truth, or all three of them. Ryan says exactly one of Vina, Otis and Morgan tells the truth.Marina says Ozzy lies. Bernita says Willian, Osian and Uri all tell the truth. Otis says Morgan lies. Opal says Martin tells the truth. Maybelle says John lies. Ocean says McKenzie lies. Morgan says Otis lies. Do Sherrie, Disha, and Shaunda tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}