{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf3789ab-df9d-551f-8d59-5b7b4d279a2c","task_key":"temporal~5fsequences--main--cf3789ab-df9d-551f-8d59-5b7b4d279a2c","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the restaurant. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the restaurant. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 11am.\\nNancy saw William buying clothes at the mall from 3pm to 5pm.\\nLeslie saw William buying a phone at the electronics store from 5pm to 7pm.\\nDavid saw William playing tennis at the tennis court from 7pm to 10pm.\\nThe restaurant was closed after 10pm.\\nBetween what times could William have gone to the restaurant?\\nOptions:\\n(A) 3pm to 5pm\\n(B) 11am to 3pm\\n(C) 5pm to 7pm\\n(D) 7pm to 10pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}