{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"581f346d-8e21-56f4-9025-4e553d697b78","task_key":"temporal~5fsequences--main--581f346d-8e21-56f4-9025-4e553d697b78","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the dance studio. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the dance studio. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 8am.\\nLinda saw Jennifer buying cookies at a bakery from 8am to 10am.\\nWilliam saw Jennifer walking in the garden from 10am to 2pm.\\nTiffany saw Jennifer waiting at the train station from 2pm to 6pm.\\nThe dance studio was closed after 10pm.\\nBetween what times could Jennifer have gone to the dance studio?\\nOptions:\\n(A) 2pm to 6pm\\n(B) 6pm to 10pm\\n(C) 8am to 10am\\n(D) 10am to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}