{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"691eef15-b1cc-5a51-a5c3-93a9e3ffd5ed","task_key":"temporal~5fsequences--main--691eef15-b1cc-5a51-a5c3-93a9e3ffd5ed","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the gas station. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the gas station. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 7am.\\nKimberly saw Sarah sitting on a rooftop from 7am to 9am.\\nMark saw Sarah buying a bike at the bike shop from 11am to 2pm.\\nJohn saw Sarah buying cookies at a bakery from 2pm to 3pm.\\nWilliam saw Sarah fixing their computer at the electronic store from 3pm to 4pm.\\nThe gas station was closed after 4pm.\\nBetween what times could Sarah have gone to the gas station?\\nOptions:\\n(A) 11am to 2pm\\n(B) 2pm to 3pm\\n(C) 9am to 11am\\n(D) 3pm to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}