{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0610739-77aa-5d35-96b0-e825f751469d","task_key":"temporal~5fsequences--main--a0610739-77aa-5d35-96b0-e825f751469d","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the coffee shop. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the coffee shop. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 5am.\\nSteven saw Jennifer stretching at a yoga studio from 5am to 8am.\\nRichard saw Jennifer waiting at the train station from 8am to 10am.\\nMark saw Jennifer walking in the garden from 12pm to 7pm.\\nThe coffee shop was closed after 7pm.\\nBetween what times could Jennifer have gone to the coffee shop?\\nOptions:\\n(A) 10am to 12pm\\n(B) 5am to 8am\\n(C) 8am to 10am\\n(D) 12pm to 7pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}