{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b22d6d9d-b4ce-53a8-8c88-eb60b8b359ec","task_key":"temporal~5fsequences--main--b22d6d9d-b4ce-53a8-8c88-eb60b8b359ec","task_revision_id":"1","upstream_id":"","short_description":"Today, Leslie went to the movies. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Leslie went to the movies. Between what times could they have gone?\\nWe know that:\\nLeslie woke up at 6am.\\nThomas saw Leslie walking in the garden from 7am to 9am.\\nNancy saw Leslie buying a bike at the bike shop from 9am to 3pm.\\nAshley saw Leslie driving to the water park from 3pm to 4pm.\\nThe movies was closed after 4pm.\\nBetween what times could Leslie have gone to the movies?\\nOptions:\\n(A) 3pm to 4pm\\n(B) 7am to 9am\\n(C) 9am to 3pm\\n(D) 6am to 7am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}