{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"059b3078-959a-5bd4-9555-dfcab7b17325","task_key":"temporal~5fsequences--main--059b3078-959a-5bd4-9555-dfcab7b17325","task_revision_id":"1","upstream_id":"","short_description":"Today, Leslie went to the orchestra hall. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Leslie went to the orchestra hall. Between what times could they have gone?\\nWe know that:\\nLeslie woke up at 7am.\\nRichard saw Leslie walking in the garden from 7am to 8am.\\nLeslie saw Leslie buying cookies at a bakery from 8am to 9am.\\nJason saw Leslie buying a bike at the bike shop from 9am to 10am.\\nSamantha saw Leslie taking photos near the Eiffel Tower from 10am to 3pm.\\nLinda saw Leslie taking photos near the Leaning Tower of Pisa from 6pm to 8pm.\\nThe orchestra hall was closed after 8pm.\\nBetween what times could Leslie have gone to the orchestra hall?\\nOptions:\\n(A) 3pm to 6pm\\n(B) 9am to 10am\\n(C) 7am to 8am\\n(D) 10am to 3pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}