{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54eabe3b-e379-580c-a190-9823e5f01ace","task_key":"temporal~5fsequences--main--54eabe3b-e379-580c-a190-9823e5f01ace","task_revision_id":"1","upstream_id":"","short_description":"Today, Anthony went to the park. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Anthony went to the park. Between what times could they have gone?\\nWe know that:\\nAnthony woke up at 6am.\\nThomas saw Anthony attending class at the school from 6am to 2pm.\\nRichard saw Anthony buying lunch at the deli from 2pm to 5pm.\\nSusan saw Anthony reading at the library from 5pm to 8pm.\\nThe park was closed after 9pm.\\nBetween what times could Anthony have gone to the park?\\nOptions:\\n(A) 2pm to 5pm\\n(B) 5pm to 8pm\\n(C) 6am to 2pm\\n(D) 8pm to 9pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}