{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e25dc6b9-2e17-55a1-a80b-1da9660ed0da","task_key":"temporal~5fsequences--main--e25dc6b9-2e17-55a1-a80b-1da9660ed0da","task_revision_id":"1","upstream_id":"","short_description":"Today, Leslie went to the swimming pool. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Leslie went to the swimming pool. Between what times could they have gone?\\nWe know that:\\nLeslie woke up at 7am.\\nWilliam saw Leslie working out at the gym from 6pm to 7pm.\\nMark saw Leslie getting a coffee at the cafe from 7pm to 8pm.\\nHannah saw Leslie taking photos near the Leaning Tower of Pisa from 8pm to 10pm.\\nThe swimming pool was closed after 10pm.\\nBetween what times could Leslie have gone to the swimming pool?\\nOptions:\\n(A) 7pm to 8pm\\n(B) 6pm to 7pm\\n(C) 7am to 6pm\\n(D) 8pm to 10pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}