{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6a4d928-244a-5c2c-8701-a69a159a2508","task_key":"temporal~5fsequences--main--b6a4d928-244a-5c2c-8701-a69a159a2508","task_revision_id":"1","upstream_id":"","short_description":"Today, Michael went to the park. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Michael went to the park. Between what times could they have gone?\\nWe know that:\\nMichael woke up at 7am.\\nWilliam saw Michael playing tennis at the tennis court from 7am to 8am.\\nRichard saw Michael working at the office from 1pm to 3pm.\\nEmily saw Michael buying lunch at the deli from 3pm to 8pm.\\nMark saw Michael walking towards the Statue of Liberty from 8pm to 10pm.\\nThe park was closed after 10pm.\\nBetween what times could Michael have gone to the park?\\nOptions:\\n(A) 3pm to 8pm\\n(B) 8am to 1pm\\n(C) 7am to 8am\\n(D) 8pm to 10pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}