{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"431f5994-256e-5be7-b7a3-6db3eeb96d2e","task_key":"temporal~5fsequences--main--431f5994-256e-5be7-b7a3-6db3eeb96d2e","task_revision_id":"1","upstream_id":"","short_description":"Today, Kimberly went to the football field. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Kimberly went to the football field. Between what times could they have gone?\\nWe know that:\\nKimberly woke up at 6am.\\nWilliam saw Kimberly buying a phone at the electronics store from 6am to 8am.\\nJason saw Kimberly waiting at the train station from 8am to 9am.\\nTiffany saw Kimberly walking towards the Statue of Liberty from 9am to 1pm.\\nLinda saw Kimberly stretching at a yoga studio from 3pm to 8pm.\\nSean saw Kimberly driving to the water park from 8pm to 9pm.\\nThe football field was closed after 9pm.\\nBetween what times could Kimberly have gone to the football field?\\nOptions:\\n(A) 8am to 9am\\n(B) 9am to 1pm\\n(C) 1pm to 3pm\\n(D) 8pm to 9pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}