{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8be00697-bd57-50eb-baf0-a491cfcb3853","task_key":"bbeh~5fweb~5fof~5flies--full--8be00697-bd57-50eb-baf0-a491cfcb3853","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Tamika says either all three of Fidel, Ozzy and Murray lie, or two of them tell the truth. Mindy says Mina lies. Phoebe says either exactly two of Willian, Oden and Maverick tell the truth or none of them. Ozzy says Murray tells the truth. Oden says Maverick tells the truth. McKenzie says Olaf lies. Amberly says Vina tells the truth. Vina says exactly two of Leda, Michelle and Melissa tell the truth. Fidel says Shaunda lies. Melissa says Michelle lies. Marina says Milton tells the truth. Matthew says Orhan lies. Lorine says exactly two of Kristian, Mindy and Mina tells the truth. Millicent says Mariah lies. Alejandro says Sherrie lies. Mason says Otis lies. Alexis says Alejandro, Merlin and Matthew all tell the truth. Otis says Mason lies. Merlin says Orhan tells the truth. Audrie says only one of Alexis, Lalit and Osian tells the truth. Oda says Nicole tells the truth. Malcolm says Odette lies. Michelle says Melissa lies. Osian says Mitchell lies. Rashida says exactly one of Antwan, Malcolm and Odette tells the truth.Antwan says exactly two of Lorine, Mariah and Millicent tells the truth. Mariah says Millicent lies. Ka says Audrie, Marina and Maria all tell the truth. Olaf says McKenzie lies. Odette says Malcolm lies. Maria says Milton lies. Mina says Mindy lies. Leda says exactly two of Rashida, Olaf and McKenzie tell the truth. Nicole says Oda tells the truth. Murray says Ozzy tells the truth. Sherrie says either exactly two of Amberly, Nicole and Oda tell the truth or none of them. Kristian says exactly two of Phoebe, Mason and Otis tells the truth. Shaunda says Ka lies. Lalit says Mitchell tells the truth. Maverick says Oden tells the truth. Willian tells the truth. Do Alejandro, Audrie, and Tamika tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}