{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90d226dd-753c-5c46-9155-5168ea0672dc","task_key":"temporal~5fsequences--main--90d226dd-753c-5c46-9155-5168ea0672dc","task_revision_id":"1","upstream_id":"","short_description":"Today, James went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, James went to the basketball court. Between what times could they have gone?\\nWe know that:\\nJames woke up at 6am.\\nSusan saw James buying a bike at the bike shop from 7am to 8am.\\nEmily saw James buying a phone at the electronics store from 8am to 9am.\\nLisa saw James attending class at the school from 9am to 3pm.\\nSarah saw James taking photos near the Leaning Tower of Pisa from 3pm to 5pm.\\nDavid saw James getting a coffee at the cafe from 5pm to 8pm.\\nThe basketball court was closed after 8pm.\\nBetween what times could James have gone to the basketball court?\\nOptions:\\n(A) 6am to 7am\\n(B) 9am to 3pm\\n(C) 8am to 9am\\n(D) 3pm to 5pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}