{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c78c81c9-b018-5111-9145-de3cac65efa7","task_key":"temporal~5fsequences--main--c78c81c9-b018-5111-9145-de3cac65efa7","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the art show. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the art show. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 5am.\\nKimberly saw Sarah waiting at the airport from 5am to 8am.\\nThomas saw Sarah taking photos near the Eiffel Tower from 8am to 1pm.\\nHannah saw Sarah fixing their computer at the electronic store from 1pm to 2pm.\\nThe art show was closed after 10pm.\\nBetween what times could Sarah have gone to the art show?\\nOptions:\\n(A) 1pm to 2pm\\n(B) 5am to 8am\\n(C) 2pm to 10pm\\n(D) 8am to 1pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}