{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"884d5a5f-1771-5f7f-8419-26b1cf5fefa2","task_key":"temporal~5fsequences--main--884d5a5f-1771-5f7f-8419-26b1cf5fefa2","task_revision_id":"1","upstream_id":"","short_description":"Today, Nancy went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Nancy went to the basketball court. Between what times could they have gone?\\nWe know that:\\nNancy woke up at 9am.\\nSteven saw Nancy waiting at the train station from 2pm to 4pm.\\nSarah saw Nancy reading at the library from 4pm to 5pm.\\nRichard saw Nancy buying a phone at the electronics store from 5pm to 6pm.\\nThomas saw Nancy attending class at the school from 6pm to 7pm.\\nThe basketball court was closed after 7pm.\\nBetween what times could Nancy have gone to the basketball court?\\nOptions:\\n(A) 9am to 2pm\\n(B) 4pm to 5pm\\n(C) 5pm to 6pm\\n(D) 2pm to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}