{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f901142e-07ac-577f-8745-845ae2750f4d","task_key":"bbeh~5fweb~5fof~5flies--full--f901142e-07ac-577f-8745-845ae2750f4d","task_revision_id":"1","upstream_id":"","short_description":"In this question, assume each person either always tells the truth or always…","config":"bbeh_web_of_lies","split":"full","body":"{\"input\":\"In this question, assume each person either always tells the truth or always lies. Charlie is at the barbershop. The person at the museum says the person at the ice skating rink lies. The person at the theater says the person at the zoo lies. The person at the farm lies. The person at the aquarium says the person at the school tells the truth. The person at the airport says the person at the ice skating rink lies. The person at the ice skating rink says the person at the theater tells the truth. Ayaan is at the museum. Ethan is at the beach. Beatriz is at the airport. The person at the ice skating rink says the person at the beach lies. Tala is at the skate park. Mateo is at the farm. Grace is at the theater. Emily is at the school. The person at the barbershop says the person at the skate park lies. The person at the skate park lies. The person at the train station says the person at the museum tells the truth. The person at the school says the person at the barbershop lies. Hiroshi is at the ice skating rink. Luna is at the train station. The person at the zoo says the person at the aquarium tells the truth. Jaxon is at the zoo. Zara is at the aquarium. Does the person at the train station tell the truth? Does the person at the museum tell the truth? Does the person at the ice skating rink tell the truth? Your answer should be a list of three words, yes or no (for example, yes, no, yes).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}