{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90439c04-c555-5c73-87a9-4fc1b96866e2","task_key":"temporal~5fsequences--main--90439c04-c555-5c73-87a9-4fc1b96866e2","task_revision_id":"1","upstream_id":"","short_description":"Today, Elizabeth went to the bakery. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Elizabeth went to the bakery. Between what times could they have gone?\\nWe know that:\\nElizabeth woke up at 8am.\\nDavid saw Elizabeth attending class at the school from 8am to 4pm.\\nElizabeth saw Elizabeth fixing their computer at the electronic store from 4pm to 5pm.\\nAshley saw Elizabeth getting a coffee at the cafe from 5pm to 9pm.\\nThe bakery was closed after 10pm.\\nBetween what times could Elizabeth have gone to the bakery?\\nOptions:\\n(A) 8am to 4pm\\n(B) 9pm to 10pm\\n(C) 4pm to 5pm\\n(D) 5pm to 9pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}