{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc90e9bc-6ffa-5b2d-83f5-bd19008c4742","task_key":"temporal~5fsequences--main--bc90e9bc-6ffa-5b2d-83f5-bd19008c4742","task_revision_id":"1","upstream_id":"","short_description":"Today, Anthony went to the amusement park. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Anthony went to the amusement park. Between what times could they have gone?\\nWe know that:\\nAnthony woke up at 6am.\\nLeslie saw Anthony reading at the library from 6am to 8am.\\nJennifer saw Anthony working out at the gym from 8am to 9am.\\nAnthony saw Anthony buying cookies at a bakery from 2pm to 9pm.\\nWilliam saw Anthony buying a bike at the bike shop from 9pm to 10pm.\\nThe amusement park was closed after 10pm.\\nBetween what times could Anthony have gone to the amusement park?\\nOptions:\\n(A) 9pm to 10pm\\n(B) 9am to 2pm\\n(C) 6am to 8am\\n(D) 2pm to 9pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}