{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8e24685-fc78-540d-af27-999a93812c29","task_key":"temporal~5fsequences--main--c8e24685-fc78-540d-af27-999a93812c29","task_revision_id":"1","upstream_id":"","short_description":"Today, Jessica went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jessica went to the basketball court. Between what times could they have gone?\\nWe know that:\\nJessica woke up at 5am.\\nMichael saw Jessica buying clothes at the mall from 5am to 6am.\\nSteven saw Jessica buying a bike at the bike shop from 6am to 7am.\\nSean saw Jessica waiting at the train station from 7am to 2pm.\\nElizabeth saw Jessica taking photos near the Eiffel Tower from 2pm to 6pm.\\nSarah saw Jessica walking towards the Statue of Liberty from 6pm to 9pm.\\nThe basketball court was closed after 10pm.\\nBetween what times could Jessica have gone to the basketball court?\\nOptions:\\n(A) 6pm to 9pm\\n(B) 5am to 6am\\n(C) 9pm to 10pm\\n(D) 2pm to 6pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}