{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0c15a83-9655-5f21-b02a-6a5a04800c20","task_key":"temporal~5fsequences--main--a0c15a83-9655-5f21-b02a-6a5a04800c20","task_revision_id":"1","upstream_id":"","short_description":"Today, David went to the art show. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, David went to the art show. Between what times could they have gone?\\nWe know that:\\nDavid woke up at 5am.\\nSamantha saw David working at the office from 5am to 7am.\\nMary saw David taking photos near the Eiffel Tower from 7am to 1pm.\\nLeslie saw David driving to the water park from 3pm to 8pm.\\nThe art show was closed after 8pm.\\nBetween what times could David have gone to the art show?\\nOptions:\\n(A) 7am to 1pm\\n(B) 3pm to 8pm\\n(C) 5am to 7am\\n(D) 1pm to 3pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}