{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6a2eaeb-f1bc-53e3-84d5-5cb31652e88c","task_key":"bbeh~5fweb~5fof~5flies--full--b6a2eaeb-f1bc-53e3-84d5-5cb31652e88c","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Ozzie says Mandy tells the truth. Ka says only one of Tamika, Mitchell and Mariana tells the truth. Omar says Oscar lies. Oscar says Omar lies. Natasha says Osric lies. Lorine says exactly one of Kristian, Melissa and Melinda tells the truth.Opal says Oda tells the truth. Tamika says either exactly one of Jamey, Mandy and Ozzie tells the truth, or all three of them. Mercedes says McKenzie tells the truth. Oprah says Oda lies. Kate says Maxim lies. Kristian says Bernita, Opal and Oprah all tell the truth. Mina says Maxim tells the truth. Alexis says Lorine tells the truth. Delbert says Delfina tells the truth. Delfina says Gwenn lies. Mandy says Ozzie tells the truth. Bahar says Octavia lies. Mitchell says Olga tells the truth. Andree tells the truth. Maya says Odessa lies. Jamey says Crista tells the truth. Onyx says Mateo tells the truth. Crista says exactly two of Amberly, Maya and Odessa tells the truth. Mariana says Olga lies. McKenzie says Mercedes tells the truth. Gwenn says Andree tells the truth. Minerva says Mateo lies. Odessa says Maya lies. Bernita says exactly one of Ryan, Bahar and Octavia tells the truth.Octavia says Bahar lies. Christie says exactly two of Ka, Natasha and Osric tell the truth. Ryan says exactly one of Dallas, Omar and Oscar tell the truth. Melissa says Melinda lies. Sima says only one of Delbert, Mina and Kate lies. Osric says Natasha lies. Dallas says Sima lies. Amberly says either exactly two of Alexis, McKenzie and Mercedes tell the truth or none of them. Elanor says Christie, Onyx and Minerva all tell the truth. Melinda says Melissa lies. Do Tamika, Octavia, and Elanor tell the truth? Your answer should be a list of three words separated by commas. Each of them must be either yes (if the person tells the truth), no (if the person lies), or unknown (if the person's truth cannot be determined). For example, <unknown, no, yes> or <yes, yes, yes>.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}