{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"daff34bf-a99b-5709-a56b-90c19cbe01b9","task_key":"temporal~5fsequences--main--daff34bf-a99b-5709-a56b-90c19cbe01b9","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the amusement park. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the amusement park. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 10am.\\nAnthony saw Sarah getting a coffee at the cafe from 11am to 12pm.\\nAndrew saw Sarah buying a phone at the electronics store from 12pm to 2pm.\\nKimberly saw Sarah driving to the water park from 2pm to 7pm.\\nSarah saw Sarah attending class at the school from 7pm to 8pm.\\nJennifer saw Sarah watching a movie at the theater from 8pm to 10pm.\\nThe amusement park was closed after 10pm.\\nBetween what times could Sarah have gone to the amusement park?\\nOptions:\\n(A) 2pm to 7pm\\n(B) 12pm to 2pm\\n(C) 10am to 11am\\n(D) 7pm to 8pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}