{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81ece01b-0df2-5a22-b5f1-92148c5f5bf8","task_key":"bbeh~5fweb~5fof~5flies--full--81ece01b-0df2-5a22-b5f1-92148c5f5bf8","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Malcolm says Miller lies. Ryan says Amberly tells the truth. Owyn says Opal tells the truth. Christie says only one of Sherrie, Obadiah and Miguel lies. Bernita lies. Mina says Martha tells the truth. Orla says Orin lies. Tamika says Vina lies. Delfina says Andree tells the truth. Kandi says exactly one of Tamika, Omar and Uri tells the truth.Marlon says Octavian lies. Obadiah says Onyx tells the truth. Raymond says Conception lies. Alejandro says Delfina lies. Willian says exactly two of Elanor, Miller and Malcolm tell the truth. Minnie says Octavian tells the truth. Octavius says Oprah lies. Alexis says Alejandro tells the truth. Sherrie says Bernita tells the truth. Kristian says Lorine tells the truth. Miguel says Onyx lies. Uri says Omar lies. Conception says either exactly one of Christie, Martha and Mina tells the truth, or all three of them. Martha says Mina tells the truth. Elanor says Kandi tells the truth. Sal says exactly two of Teressa, Oprah and Octavius tells the truth. Andree says exactly one of Crista, Orin and Orla tells the truth.Lorine says only one of Jaymie, Minnie and Marlon lies. Gwenn says only one of Willian, Owyn and Oakleigh lies. Miller says Malcolm lies. Oliver says Ozzie lies. Jaymie says Gwenn tells the truth. Oakleigh says Opal lies. Crista says Kristian lies. Oscar says Maxwell lies. Omar says Uri lies. Maxwell says Oscar lies. Vina says Ryan tells the truth. Amberly says exactly two of Raymond, Ozzie and Oliver tell the truth. Orin says Orla lies. Teressa says exactly two of Alexis, Maxwell and Oscar tells the truth. Ozzie says Oliver lies. Oprah says Octavius lies. Do Alexis, Miller, and Sal tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}