{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"533b5166-1204-584a-a094-4d178b8ad827","task_key":"temporal~5fsequences--main--533b5166-1204-584a-a094-4d178b8ad827","task_revision_id":"1","upstream_id":"","short_description":"Today, Jessica went to the bookstore. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jessica went to the bookstore. Between what times could they have gone?\\nWe know that:\\nJessica woke up at 6am.\\nSteven saw Jessica working out at the gym from 7am to 2pm.\\nJason saw Jessica taking photos near the Leaning Tower of Pisa from 2pm to 3pm.\\nEmily saw Jessica taking photos near the Eiffel Tower from 3pm to 9pm.\\nThe bookstore was closed after 9pm.\\nBetween what times could Jessica have gone to the bookstore?\\nOptions:\\n(A) 2pm to 3pm\\n(B) 6am to 7am\\n(C) 7am to 2pm\\n(D) 3pm to 9pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}