{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f3fe2c2-2963-51cf-b2f9-8c53ee4f1f74","task_key":"temporal~5fsequences--main--9f3fe2c2-2963-51cf-b2f9-8c53ee4f1f74","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the art show. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the art show. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 8am.\\nLisa saw Jennifer buying clothes at the mall from 8am to 9am.\\nSteven saw Jennifer walking towards the Statue of Liberty from 9am to 10am.\\nSarah saw Jennifer sitting on a rooftop from 10am to 2pm.\\nLinda saw Jennifer taking photos near the Leaning Tower of Pisa from 4pm to 6pm.\\nAndrew saw Jennifer working out at the gym from 6pm to 8pm.\\nThe art show was closed after 8pm.\\nBetween what times could Jennifer have gone to the art show?\\nOptions:\\n(A) 2pm to 4pm\\n(B) 8am to 9am\\n(C) 9am to 10am\\n(D) 6pm to 8pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}