{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"924cb675-31b4-5e4e-b58e-603b6629721d","task_key":"temporal~5fsequences--main--924cb675-31b4-5e4e-b58e-603b6629721d","task_revision_id":"1","upstream_id":"","short_description":"Today, David went to the coffee shop. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, David went to the coffee shop. Between what times could they have gone?\\nWe know that:\\nDavid woke up at 8am.\\nElizabeth saw David waiting at the airport from 10am to 1pm.\\nDavid saw David playing tennis at the tennis court from 1pm to 2pm.\\nJessica saw David working out at the gym from 2pm to 3pm.\\nSusan saw David waiting at the train station from 3pm to 4pm.\\nSean saw David sitting on a rooftop from 4pm to 6pm.\\nThe coffee shop was closed after 6pm.\\nBetween what times could David have gone to the coffee shop?\\nOptions:\\n(A) 3pm to 4pm\\n(B) 10am to 1pm\\n(C) 8am to 10am\\n(D) 1pm to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}