{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99ac6604-f4ad-5e0d-bcb0-035e94801b50","task_key":"bbeh~5fweb~5fof~5flies--full--99ac6604-f4ad-5e0d-bcb0-035e94801b50","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Delbert says Alexis, Mehran and Oakley all tell the truth. Mallory says Oswin lies. Octavius says Owyn tells the truth. Minerva says Osborne lies. Mason says Malcolm tells the truth. Mateo says Oona lies. Oden says Matteo tells the truth. Leda says either exactly one of Vina, Milton and Muhammad tells the truth, or all three of them. Mohamed says Marcus lies. Miguel says Mariah tells the truth. Lorine says either exactly one of Tamika, Otis and Millie tells the truth, or all three of them. Matteo says Oden tells the truth. Jamey says Sal, Marsha and Olga all tell the truth. Nicole says Ottoline lies. Ottoline says Nicole lies. Mackenzie says Lalit tells the truth. Willian says Delbert lies. Bernita says exactly two of Ryan, Octavia and Meredith tells the truth. Malcolm says Mason tells the truth. Alexis says exactly one of Yoland, Maya and Murphy tells the truth.Octavia says Meredith lies. Otis says Millie tells the truth. McKenzie says Mia lies. Ka says either all three of Lorine, Lalit and Mackenzie lie, or two of them tell the truth. Odile says Orion tells the truth. Shalonda says either exactly two of Willian, Octavio and Olaf tell the truth or none of them. Michelle says Mary tells the truth. Octavian says Meadow tells the truth. Marcus says Mohamed lies. Octavio says Olaf tells the truth. Oakley says Ocean lies. Lalit says Mackenzie tells the truth. Fletcher lies. Otto says Oscar lies. Milton says Muhammad tells the truth. Maya says Murphy lies. Mary says Michelle tells the truth. Olaf says Octavio tells the truth. Oscar says Otto lies. Raymond says exactly one of Shalonda, Mohamed and Marcus tells the truth.Mehran says Ocean tells the truth. Megan says Mia tells the truth. Antwan says either all three of Leda, Matteo and Oden lie, or two of them tell the truth. Gwenn says exactly two of Phoebe, Osborne and Minerva tells the truth. Odessa says Meadow lies. Vina says either exactly two of Raymond, Mariah and Miguel tell the truth or none of them. Orion says Odile tells the truth. Ryan says only one of Fletcher, Octavius and Max lies. Christie says either all three of Ka, Mason and Malcolm tell the truth or only one of them. Meredith says Octavia lies. Marsha says Margaret tells the truth. Osborne says Minerva lies. Olga says Margaret lies. Millie says Otis tells the truth. Othello says Oswin tells the truth. Murphy says Maya lies. Max says Owyn lies. Yoland says either exactly two of Shenna, Odile and Orion tell the truth or none of them. Helene says only one of Conception, Megan and McKenzie tells the truth. Phoebe says only one of Jamey, Octavian and Odessa lies. Muhammad says Milton tells the truth. Mariah says Miguel tells the truth. Shenna says only one of Helene, Othello and Mallory lies. Conception says exactly one of Christie, Otto and Oscar tell the truth. Sima says exactly two of Gwenn, Mateo and Oona tells the truth. Oona says Mateo lies. Tamika says exactly two of Sima, Ottoline and Nicole tells the truth. Sal says either all three of Bernita, Michelle and Mary lie, or two of them tell the truth. Do Ka, Raymond, and Antwan tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}