{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64c1cbca-9543-5bb4-b42d-26d8d2f8db06","task_key":"temporal~5fsequences--main--64c1cbca-9543-5bb4-b42d-26d8d2f8db06","task_revision_id":"1","upstream_id":"","short_description":"Today, Emily went to the park. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Emily went to the park. Between what times could they have gone?\\nWe know that:\\nEmily woke up at 5am.\\nLeslie saw Emily getting a coffee at the cafe from 5am to 6am.\\nKimberly saw Emily walking towards the Statue of Liberty from 6am to 2pm.\\nRichard saw Emily waiting at the airport from 4pm to 10pm.\\nThe park was closed after 10pm.\\nBetween what times could Emily have gone to the park?\\nOptions:\\n(A) 6am to 2pm\\n(B) 4pm to 10pm\\n(C) 5am to 6am\\n(D) 2pm to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}