{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e2fe9f5-e228-5369-874b-6cab6aeb3971","task_key":"temporal~5fsequences--main--7e2fe9f5-e228-5369-874b-6cab6aeb3971","task_revision_id":"1","upstream_id":"","short_description":"Today, Andrew went to the art show. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Andrew went to the art show. Between what times could they have gone?\\nWe know that:\\nAndrew woke up at 6am.\\nKimberly saw Andrew waiting at the airport from 10am to 12pm.\\nAnthony saw Andrew attending class at the school from 12pm to 7pm.\\nJohn saw Andrew working out at the gym from 7pm to 8pm.\\nThe art show was closed after 8pm.\\nBetween what times could Andrew have gone to the art show?\\nOptions:\\n(A) 6am to 10am\\n(B) 7pm to 8pm\\n(C) 12pm to 7pm\\n(D) 10am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}