{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b36d71d3-454c-53ce-bfe7-83a8af729be4","task_key":"temporal~5fsequences--main--b36d71d3-454c-53ce-bfe7-83a8af729be4","task_revision_id":"1","upstream_id":"","short_description":"Today, Emily went to the orchestra hall. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Emily went to the orchestra hall. Between what times could they have gone?\\nWe know that:\\nEmily woke up at 5am.\\nLinda saw Emily waiting at the airport from 5am to 6am.\\nDavid saw Emily buying cookies at a bakery from 2pm to 7pm.\\nMichael saw Emily buying clothes at the mall from 7pm to 8pm.\\nThomas saw Emily working out at the gym from 8pm to 10pm.\\nThe orchestra hall was closed after 10pm.\\nBetween what times could Emily have gone to the orchestra hall?\\nOptions:\\n(A) 5am to 6am\\n(B) 2pm to 7pm\\n(C) 8pm to 10pm\\n(D) 6am to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}