{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19afe18b-031c-5f97-a658-3b31119ee680","task_key":"temporal~5fsequences--main--19afe18b-031c-5f97-a658-3b31119ee680","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the construction site. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the construction site. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 6am.\\nNancy saw Sarah fixing their computer at the electronic store from 8am to 2pm.\\nBetty saw Sarah driving to the water park from 2pm to 3pm.\\nKimberly saw Sarah buying clothes at the mall from 3pm to 4pm.\\nEmily saw Sarah buying cookies at a bakery from 4pm to 6pm.\\nDavid saw Sarah attending class at the school from 6pm to 8pm.\\nThe construction site was closed after 8pm.\\nBetween what times could Sarah have gone to the construction site?\\nOptions:\\n(A) 3pm to 4pm\\n(B) 2pm to 3pm\\n(C) 6am to 8am\\n(D) 8am to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}