{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30698b5c-1eb7-59bb-9011-d0165edd5e2b","task_key":"temporal~5fsequences--main--30698b5c-1eb7-59bb-9011-d0165edd5e2b","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the art studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the art studio. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 6am.\\nAnthony saw William getting a coffee at the cafe from 6am to 7am.\\nJohn saw William watching a movie at the theater from 8am to 10am.\\nHannah saw William waiting at the train station from 10am to 12pm.\\nMark saw William walking towards the Statue of Liberty from 12pm to 1pm.\\nWilliam saw William working out at the gym from 1pm to 6pm.\\nThe art studio was closed after 6pm.\\nBetween what times could William have gone to the art studio?\\nOptions:\\n(A) 8am to 10am\\n(B) 7am to 8am\\n(C) 1pm to 6pm\\n(D) 12pm to 1pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}