{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce4c3ee0-6c89-5d84-9179-b536f39d19cd","task_key":"temporal~5fsequences--main--ce4c3ee0-6c89-5d84-9179-b536f39d19cd","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the art studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the art studio. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 8am.\\nSteven saw William working at the office from 8am to 10am.\\nDavid saw William working out at the gym from 10am to 3pm.\\nKimberly saw William taking photos near the Leaning Tower of Pisa from 3pm to 5pm.\\nMary saw William buying clothes at the mall from 6pm to 8pm.\\nSean saw William taking photos near the Eiffel Tower from 8pm to 10pm.\\nThe art studio was closed after 10pm.\\nBetween what times could William have gone to the art studio?\\nOptions:\\n(A) 8pm to 10pm\\n(B) 6pm to 8pm\\n(C) 10am to 3pm\\n(D) 5pm to 6pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}