{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c32d524-aab2-5742-a97d-379d361ff0c8","task_key":"temporal~5fsequences--main--9c32d524-aab2-5742-a97d-379d361ff0c8","task_revision_id":"1","upstream_id":"","short_description":"Today, Mark went to the football field. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Mark went to the football field. Between what times could they have gone?\\nWe know that:\\nMark woke up at 5am.\\nSteven saw Mark walking towards the Statue of Liberty from 5am to 6am.\\nWilliam saw Mark working out at the gym from 6am to 7am.\\nJennifer saw Mark reading at the library from 7am to 9pm.\\nThe football field was closed after 10pm.\\nBetween what times could Mark have gone to the football field?\\nOptions:\\n(A) 7am to 9pm\\n(B) 6am to 7am\\n(C) 5am to 6am\\n(D) 9pm to 10pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}