{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29c87427-8039-50eb-a1f1-c0f1b1f18bff","task_key":"temporal~5fsequences--main--29c87427-8039-50eb-a1f1-c0f1b1f18bff","task_revision_id":"1","upstream_id":"","short_description":"Today, Samantha went to the amusement park. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Samantha went to the amusement park. Between what times could they have gone?\\nWe know that:\\nSamantha woke up at 5am.\\nHannah saw Samantha buying a phone at the electronics store from 5am to 7am.\\nJames saw Samantha working out at the gym from 7am to 8am.\\nRichard saw Samantha working at the office from 8am to 11am.\\nMark saw Samantha buying lunch at the deli from 11am to 2pm.\\nWilliam saw Samantha buying a bike at the bike shop from 7pm to 8pm.\\nThe amusement park was closed after 8pm.\\nBetween what times could Samantha have gone to the amusement park?\\nOptions:\\n(A) 2pm to 7pm\\n(B) 11am to 2pm\\n(C) 5am to 7am\\n(D) 7am to 8am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}