{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"547ee12b-3848-55cd-8581-abb7cc3e5acd","task_key":"bbeh~5fweb~5fof~5flies--full--547ee12b-3848-55cd-8581-abb7cc3e5acd","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Kehinde is at the park. The person at the hotel tells the truth. Max is at the restaurant. The person at the hotel says the person at the art gallery tells the truth. The person at the amusement park says the person at the library tells the truth. Theo is at the ice skating rink. Zara is at the observatory. Farid is at the movie theater. Charlie is at the train station. The person at the airport thinks their neighbor is telling the truth. The person at the school says the person at the hotel tells the truth. Mateo is at the school. The person at the beach says the person at the amusement park lies. Ethan is at the library. The person at the park lies. The person at the bowling alley says the person at the beach lies. The person at the barbershop lies. Elowen is at the campground. Olivia is at the hotel. The person at the campground says the person at the airport tells the truth. The person at the school thinks their neighbor is telling the truth. The person at the airport says the person at the school tells the truth. Jaxon is at the amusement park. Grace is at the beach. Priya is at the gym. The person at the ice skating rink says the person at the school lies. Jake is at the barbershop. The person at the restaurant says the person at the airport tells the truth. The person at the school says the person at the movie theater tells the truth. Devika is at the bowling alley. The person at the library says the person at the restaurant lies. Tala is at the art gallery. The person at the observatory says the person at the library tells the truth. The person at the gym tells the truth. Hiroshi is at the airport. The person at the train station says the person at the amusement park lies. Does the person at the beach tell the truth? Does the person at the amusement park tell the truth? Does the person at the library tell the truth? Your answer should be a list of three words, yes or no (for example, yes, no, yes).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}