{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e0f10be-a38c-5d00-b351-78e5252b181d","task_key":"temporal~5fsequences--main--3e0f10be-a38c-5d00-b351-78e5252b181d","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the art show. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the art show. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 11am.\\nLisa saw Sarah working at the office from 11am to 2pm.\\nHannah saw Sarah reading at the library from 2pm to 4pm.\\nElizabeth saw Sarah fixing their computer at the electronic store from 4pm to 7pm.\\nThe art show was closed after 9pm.\\nBetween what times could Sarah have gone to the art show?\\nOptions:\\n(A) 11am to 2pm\\n(B) 7pm to 9pm\\n(C) 2pm to 4pm\\n(D) 4pm to 7pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}