{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"539225b7-4b82-5ba2-97d1-2c1156b14cbb","task_key":"temporal~5fsequences--main--539225b7-4b82-5ba2-97d1-2c1156b14cbb","task_revision_id":"1","upstream_id":"","short_description":"Today, Emily went to the bakery. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Emily went to the bakery. Between what times could they have gone?\\nWe know that:\\nEmily woke up at 5am.\\nNancy saw Emily sitting on a rooftop from 6am to 7am.\\nBetty saw Emily reading at the library from 7am to 8am.\\nJason saw Emily buying lunch at the deli from 8am to 6pm.\\nDavid saw Emily attending class at the school from 6pm to 7pm.\\nSamantha saw Emily driving to the water park from 7pm to 10pm.\\nThe bakery was closed after 10pm.\\nBetween what times could Emily have gone to the bakery?\\nOptions:\\n(A) 5am to 6am\\n(B) 8am to 6pm\\n(C) 6pm to 7pm\\n(D) 7am to 8am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}