{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86334198-378c-5f0e-9ab0-e75c27485b3a","task_key":"temporal~5fsequences--main--86334198-378c-5f0e-9ab0-e75c27485b3a","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the swimming pool. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the swimming pool. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 5am.\\nJennifer saw William walking in the garden from 7am to 5pm.\\nMary saw William reading at the library from 5pm to 6pm.\\nWilliam saw William buying lunch at the deli from 6pm to 10pm.\\nThe swimming pool was closed after 10pm.\\nBetween what times could William have gone to the swimming pool?\\nOptions:\\n(A) 5pm to 6pm\\n(B) 6pm to 10pm\\n(C) 5am to 7am\\n(D) 7am to 5pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}