{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9d7bb88-8d04-5154-aa73-349f1053c580","task_key":"bbeh~5fweb~5fof~5flies--full--b9d7bb88-8d04-5154-aa73-349f1053c580","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. The person at the school tells the truth. Maya is at the city hall. The person at the beach says the person at the barbershop tells the truth. Jake is at the bookstore. Hiroshi is at the skate park. The person at the city hall says the person at the amusement park lies. Soren is at the farm. The person at the cafe says the person at the skate park tells the truth. Tala is at the barbershop. Ayaan is at the aquarium. The person at the museum tells the truth. The person at the bowling alley says the person at the aquarium tells the truth. The person at the planetarium says the person at the train station tells the truth. The person at the skate park says the person at the art gallery lies. The person at the vineyard tells the truth. Liam is at the ice skating rink. The person at the airport tells the truth. The person at the barbershop says the person at the skate park tells the truth. Priya is at the train station. Isabella is at the amusement park. The person at the aquarium says the person at the museum tells the truth. Luna is at the museum. Charlie is at the vineyard. Ethan is at the airport. The person at the ice skating rink says the person at the aquarium tells the truth. The person at the amusement park says the person at the planetarium lies. The person at the barbershop says the person at the bookstore lies. Mateo is at the cafe. Nia is at the restaurant. Kehinde is at the art gallery. The person at the train station says the person at the barbershop tells the truth. Jaxon is at the planetarium. The person at the farm says the person at the art gallery tells the truth. The person at the art gallery says the person at the bowling alley lies. Grace is at the beach. Quan is at the school. Zara is at the bowling alley. The person at the restaurant tells the truth. Does the person at the amusement park tell the truth? Does the person at the planetarium tell the truth? Does the person at the train station tell the truth? Your answer should be a list of three words, yes or no (for example, yes, no, yes).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}