{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5c1f1a7-9a06-58c4-9d0a-e16800b347ac","task_key":"temporal~5fsequences--main--d5c1f1a7-9a06-58c4-9d0a-e16800b347ac","task_revision_id":"1","upstream_id":"","short_description":"Today, David went to the football field. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, David went to the football field. Between what times could they have gone?\\nWe know that:\\nDavid woke up at 5am.\\nSteven saw David buying a phone at the electronics store from 5am to 6am.\\nKimberly saw David taking photos near the Leaning Tower of Pisa from 6am to 8am.\\nBetty saw David working out at the gym from 9am to 12pm.\\nLisa saw David driving to the water park from 12pm to 3pm.\\nWilliam saw David sitting on a rooftop from 3pm to 10pm.\\nThe football field was closed after 10pm.\\nBetween what times could David have gone to the football field?\\nOptions:\\n(A) 6am to 8am\\n(B) 5am to 6am\\n(C) 3pm to 10pm\\n(D) 8am to 9am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}