{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f5b30de-e851-5d9e-91a7-434d7a1429ed","task_key":"temporal~5fsequences--main--9f5b30de-e851-5d9e-91a7-434d7a1429ed","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the bookstore. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the bookstore. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 7am.\\nTiffany saw William working out at the gym from 7am to 8am.\\nSean saw William driving to the water park from 12pm to 1pm.\\nAnthony saw William waiting at the train station from 1pm to 3pm.\\nJames saw William taking photos near the Eiffel Tower from 3pm to 5pm.\\nJessica saw William working at the office from 5pm to 7pm.\\nThe bookstore was closed after 7pm.\\nBetween what times could William have gone to the bookstore?\\nOptions:\\n(A) 12pm to 1pm\\n(B) 1pm to 3pm\\n(C) 3pm to 5pm\\n(D) 8am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}