{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4db508c-0d1e-5ed0-8e6c-3529e1d190d8","task_key":"temporal~5fsequences--main--f4db508c-0d1e-5ed0-8e6c-3529e1d190d8","task_revision_id":"1","upstream_id":"","short_description":"Today, James went to the beach. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, James went to the beach. Between what times could they have gone?\\nWe know that:\\nJames woke up at 5am.\\nSean saw James walking towards the Statue of Liberty from 5am to 6am.\\nMichael saw James driving to the water park from 6am to 7am.\\nAnthony saw James reading at the library from 7am to 3pm.\\nWilliam saw James getting a coffee at the cafe from 4pm to 9pm.\\nThe beach was closed after 9pm.\\nBetween what times could James have gone to the beach?\\nOptions:\\n(A) 7am to 3pm\\n(B) 5am to 6am\\n(C) 4pm to 9pm\\n(D) 3pm to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}