{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b70d0f20-790a-5afb-b049-1644995e3797","task_key":"temporal~5fsequences--main--b70d0f20-790a-5afb-b049-1644995e3797","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the art studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the art studio. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 6am.\\nBetty saw William reading at the library from 12pm to 3pm.\\nHannah saw William fixing their computer at the electronic store from 3pm to 4pm.\\nNancy saw William buying cookies at a bakery from 4pm to 7pm.\\nSarah saw William walking towards the Statue of Liberty from 7pm to 8pm.\\nMichael saw William watching a movie at the theater from 8pm to 10pm.\\nThe art studio was closed after 10pm.\\nBetween what times could William have gone to the art studio?\\nOptions:\\n(A) 7pm to 8pm\\n(B) 6am to 12pm\\n(C) 8pm to 10pm\\n(D) 12pm to 3pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}