{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02ab7974-e701-5b7d-b49c-40775430fcce","task_key":"temporal~5fsequences--main--02ab7974-e701-5b7d-b49c-40775430fcce","task_revision_id":"1","upstream_id":"","short_description":"Today, Andrew went to the dance studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Andrew went to the dance studio. Between what times could they have gone?\\nWe know that:\\nAndrew woke up at 8am.\\nThomas saw Andrew playing tennis at the tennis court from 8am to 11am.\\nMary saw Andrew reading at the library from 11am to 1pm.\\nMichael saw Andrew fixing their computer at the electronic store from 8pm to 10pm.\\nThe dance studio was closed after 10pm.\\nBetween what times could Andrew have gone to the dance studio?\\nOptions:\\n(A) 8pm to 10pm\\n(B) 11am to 1pm\\n(C) 8am to 11am\\n(D) 1pm to 8pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}