{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3803fb37-7589-5afc-9b30-39ab39d86041","task_key":"temporal~5fsequences--main--3803fb37-7589-5afc-9b30-39ab39d86041","task_revision_id":"1","upstream_id":"","short_description":"Today, Emily went to the dance studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Emily went to the dance studio. Between what times could they have gone?\\nWe know that:\\nEmily woke up at 5am.\\nThomas saw Emily reading at the library from 5am to 7am.\\nTiffany saw Emily walking towards the Statue of Liberty from 7am to 11am.\\nJohn saw Emily playing tennis at the tennis court from 11am to 12pm.\\nSean saw Emily taking photos near the Eiffel Tower from 12pm to 8pm.\\nJason saw Emily waiting at the train station from 9pm to 10pm.\\nThe dance studio was closed after 10pm.\\nBetween what times could Emily have gone to the dance studio?\\nOptions:\\n(A) 5am to 7am\\n(B) 8pm to 9pm\\n(C) 9pm to 10pm\\n(D) 11am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}