{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49f73209-f192-5317-a10f-1b8b4404cfc4","task_key":"temporal~5fsequences--main--49f73209-f192-5317-a10f-1b8b4404cfc4","task_revision_id":"1","upstream_id":"","short_description":"Today, Mary went to the restaurant. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Mary went to the restaurant. Between what times could they have gone?\\nWe know that:\\nMary woke up at 9am.\\nDavid saw Mary walking in the garden from 9am to 11am.\\nSamantha saw Mary getting a coffee at the cafe from 2pm to 3pm.\\nSarah saw Mary driving to the water park from 3pm to 7pm.\\nThe restaurant was closed after 7pm.\\nBetween what times could Mary have gone to the restaurant?\\nOptions:\\n(A) 3pm to 7pm\\n(B) 9am to 11am\\n(C) 2pm to 3pm\\n(D) 11am to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}