{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96b30679-826f-5b23-9f62-9b6ccf955ab1","task_key":"temporal~5fsequences--main--96b30679-826f-5b23-9f62-9b6ccf955ab1","task_revision_id":"1","upstream_id":"","short_description":"Today, Leslie went to the beach. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Leslie went to the beach. Between what times could they have gone?\\nWe know that:\\nLeslie woke up at 7am.\\nAshley saw Leslie stretching at a yoga studio from 7am to 9am.\\nAndrew saw Leslie playing tennis at the tennis court from 9am to 11am.\\nJason saw Leslie walking towards the Statue of Liberty from 11am to 4pm.\\nThe beach was closed after 8pm.\\nBetween what times could Leslie have gone to the beach?\\nOptions:\\n(A) 9am to 11am\\n(B) 4pm to 8pm\\n(C) 7am to 9am\\n(D) 11am to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}