{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ed89d22-4a65-567b-a5c7-8306b56a2782","task_key":"bbeh~5fweb~5fof~5flies--full--7ed89d22-4a65-567b-a5c7-8306b56a2782","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Ozzie says Marvin tells the truth. Mina says Murray tells the truth. Vina says only one of Bernita, Sanket and Mike lies. Maria says Oran lies. Olive says Monica lies. Lorine says only one of Kandi, Bahar and Murphy tells the truth. Maximilian says Melanie tells the truth. Bernita says Dallas tells the truth. Marvin says Ozzie tells the truth. Mackenzie says Moses tells the truth. Sima says only one of Ryan, Mina and Hritik lies. Shaunda tells the truth. Ofelia says Michelle lies. Mitchell says Mindy tells the truth. Martin says Osborne lies. Sanket says Michael tells the truth. Mavis says Michelle tells the truth. Melissa says Maxim lies. Melanie says Maximilian tells the truth. Mindy says Mitchell tells the truth. Alejandro says only one of Audrie, Micah and Opal tells the truth. Odile says Melody lies. Micah says Orin tells the truth. Amberly says exactly two of Vernell, Osborne and Martin tell the truth. Maverick says Meadow lies. Ryan says exactly one of Crista, Maxim and Melissa tells the truth.Christie says either exactly one of Sherrie, Melanie and Maximilian tells the truth, or all three of them. Orly says Melody tells the truth. Hritik says Murray lies. Olaf says Oona tells the truth. Monica says Olive lies. Jaymie says exactly one of Amberly, Odysseus and Odell tell the truth. Octavian says Morris tells the truth. Vernell says Lorine tells the truth. Delfina says either exactly two of Gwenn, Olaf and Oona tell the truth or none of them. Oran says Maria lies. Mike says Michael lies. Kandi says either exactly one of Delbert, Moses and Mackenzie tells the truth, or all three of them. Dallas says Tamika, Orly and Odile all tell the truth. Murphy says Miriam lies. Gwenn says exactly one of Conception, Olive and Monica tells the truth.Odell says Odysseus lies. Crista says either exactly two of Shaunda, Mitchell and Mindy tell the truth or none of them. Shalonda says either all three of Christie, Ozzie and Marvin lie, or two of them tell the truth. Bahar says Miriam tells the truth. Ozzy says Octavio lies. Teressa says Jerry tells the truth. Octavio says Ozzy lies. Jerry says Shalonda, Mavis and Ofelia all tell the truth. Osborne says Martin lies. Maxim says Melissa lies. Oona says Olaf tells the truth. Sherrie says either exactly one of Jaymie, Morris and Octavian tells the truth, or all three of them. Moses says Mackenzie tells the truth. Opal says Orin lies. Tamika says exactly one of Teressa, Meadow and Maverick tells the truth.Odysseus says Odell lies. Morris says Octavian tells the truth. Meadow says Maverick lies. Audrie says Sima lies. Delbert says exactly one of Alejandro, Octavio and Ozzy tell the truth. Conception says exactly two of Vina, Oran and Maria tells the truth. Do Sherrie, Shalonda, and Delfina tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}