{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47c0e7a9-5c41-59a9-8894-4d1bd4a7bca0","task_key":"temporal~5fsequences--main--47c0e7a9-5c41-59a9-8894-4d1bd4a7bca0","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the orchestra hall. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the orchestra hall. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 6am.\\nRichard saw Jennifer driving to the water park from 7am to 10am.\\nSusan saw Jennifer playing tennis at the tennis court from 10am to 1pm.\\nLisa saw Jennifer waiting at the train station from 1pm to 3pm.\\nThe orchestra hall was closed after 3pm.\\nBetween what times could Jennifer have gone to the orchestra hall?\\nOptions:\\n(A) 6am to 7am\\n(B) 10am to 1pm\\n(C) 7am to 10am\\n(D) 1pm to 3pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}