{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"37feaf18-718f-5d0d-b309-31afbe6f6af8","task_key":"temporal~5fsequences--main--37feaf18-718f-5d0d-b309-31afbe6f6af8","task_revision_id":"1","upstream_id":"","short_description":"Today, Steven went to the gas station. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Steven went to the gas station. Between what times could they have gone?\\nWe know that:\\nSteven woke up at 7am.\\nAshley saw Steven stretching at a yoga studio from 7am to 10am.\\nWilliam saw Steven reading at the library from 1pm to 2pm.\\nJennifer saw Steven working at the office from 2pm to 3pm.\\nMichael saw Steven watching a movie at the theater from 3pm to 7pm.\\nMary saw Steven waiting at the train station from 7pm to 10pm.\\nThe gas station was closed after 10pm.\\nBetween what times could Steven have gone to the gas station?\\nOptions:\\n(A) 7am to 10am\\n(B) 7pm to 10pm\\n(C) 10am to 1pm\\n(D) 3pm to 7pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}