{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ac7ccc2-1271-57be-b51e-e41f78986835","task_key":"temporal~5fsequences--main--3ac7ccc2-1271-57be-b51e-e41f78986835","task_revision_id":"1","upstream_id":"","short_description":"Today, Andrew went to the park. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Andrew went to the park. Between what times could they have gone?\\nWe know that:\\nAndrew woke up at 6am.\\nDavid saw Andrew stretching at a yoga studio from 6am to 7am.\\nLeslie saw Andrew buying lunch at the deli from 7am to 9am.\\nJames saw Andrew waiting at the train station from 9am to 12pm.\\nTiffany saw Andrew working at the office from 12pm to 3pm.\\nMark saw Andrew waiting at the airport from 3pm to 7pm.\\nThe park was closed after 10pm.\\nBetween what times could Andrew have gone to the park?\\nOptions:\\n(A) 6am to 7am\\n(B) 9am to 12pm\\n(C) 7pm to 10pm\\n(D) 7am to 9am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}