{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"920c596d-eb7f-56d8-a237-cd94bea3aa54","task_key":"temporal~5fsequences--main--920c596d-eb7f-56d8-a237-cd94bea3aa54","task_revision_id":"1","upstream_id":"","short_description":"Today, Leslie went to the beach. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Leslie went to the beach. Between what times could they have gone?\\nWe know that:\\nLeslie woke up at 5am.\\nEmily saw Leslie attending class at the school from 5am to 9am.\\nAnthony saw Leslie watching a movie at the theater from 9am to 11am.\\nThomas saw Leslie buying cookies at a bakery from 11am to 4pm.\\nAshley saw Leslie fixing their computer at the electronic store from 4pm to 5pm.\\nSusan saw Leslie walking towards the Statue of Liberty from 9pm to 10pm.\\nThe beach was closed after 10pm.\\nBetween what times could Leslie have gone to the beach?\\nOptions:\\n(A) 5pm to 9pm\\n(B) 9am to 11am\\n(C) 9pm to 10pm\\n(D) 4pm to 5pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}