{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a895b1c3-0090-55e9-a54c-5ef200b72454","task_key":"temporal~5fsequences--main--a895b1c3-0090-55e9-a54c-5ef200b72454","task_revision_id":"1","upstream_id":"","short_description":"Today, Michael went to the football field. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Michael went to the football field. Between what times could they have gone?\\nWe know that:\\nMichael woke up at 5am.\\nKimberly saw Michael stretching at a yoga studio from 5am to 7am.\\nWilliam saw Michael walking in the garden from 7am to 8am.\\nSarah saw Michael fixing their computer at the electronic store from 8am to 10am.\\nNancy saw Michael waiting at the airport from 10am to 12pm.\\nSamantha saw Michael working out at the gym from 1pm to 9pm.\\nThe football field was closed after 9pm.\\nBetween what times could Michael have gone to the football field?\\nOptions:\\n(A) 12pm to 1pm\\n(B) 10am to 12pm\\n(C) 7am to 8am\\n(D) 5am to 7am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}