{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba769f18-8d2e-5091-85a9-23f5075ebfc8","task_key":"temporal~5fsequences--main--ba769f18-8d2e-5091-85a9-23f5075ebfc8","task_revision_id":"1","upstream_id":"","short_description":"Today, Hannah went to the art show. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Hannah went to the art show. Between what times could they have gone?\\nWe know that:\\nHannah woke up at 7am.\\nMichael saw Hannah walking towards the Statue of Liberty from 7am to 8am.\\nLeslie saw Hannah reading at the library from 8am to 9am.\\nJason saw Hannah sitting on a rooftop from 9am to 1pm.\\nLisa saw Hannah fixing their computer at the electronic store from 7pm to 9pm.\\nHannah saw Hannah watching a movie at the theater from 9pm to 10pm.\\nThe art show was closed after 10pm.\\nBetween what times could Hannah have gone to the art show?\\nOptions:\\n(A) 7pm to 9pm\\n(B) 7am to 8am\\n(C) 1pm to 7pm\\n(D) 9pm to 10pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}