{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe7e4e00-c91b-595f-8e11-cd05026c654d","task_key":"temporal~5fsequences--main--fe7e4e00-c91b-595f-8e11-cd05026c654d","task_revision_id":"1","upstream_id":"","short_description":"Today, Anthony went to the orchestra hall. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Anthony went to the orchestra hall. Between what times could they have gone?\\nWe know that:\\nAnthony woke up at 6am.\\nSamantha saw Anthony stretching at a yoga studio from 6am to 12pm.\\nNancy saw Anthony driving to the water park from 12pm to 2pm.\\nDavid saw Anthony watching a movie at the theater from 2pm to 3pm.\\nWilliam saw Anthony fixing their computer at the electronic store from 5pm to 6pm.\\nSarah saw Anthony reading at the library from 6pm to 7pm.\\nThe orchestra hall was closed after 7pm.\\nBetween what times could Anthony have gone to the orchestra hall?\\nOptions:\\n(A) 12pm to 2pm\\n(B) 6pm to 7pm\\n(C) 3pm to 5pm\\n(D) 2pm to 3pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}