{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99c61313-2694-5021-9b10-b6581dbfe83c","task_key":"temporal~5fsequences--main--99c61313-2694-5021-9b10-b6581dbfe83c","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the clothing store. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the clothing store. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 9am.\\nWilliam saw Sarah buying a bike at the bike shop from 9am to 12pm.\\nEmily saw Sarah waiting at the airport from 12pm to 1pm.\\nJennifer saw Sarah taking photos near the Eiffel Tower from 2pm to 5pm.\\nJason saw Sarah driving to the water park from 5pm to 6pm.\\nThe clothing store was closed after 6pm.\\nBetween what times could Sarah have gone to the clothing store?\\nOptions:\\n(A) 1pm to 2pm\\n(B) 5pm to 6pm\\n(C) 9am to 12pm\\n(D) 12pm to 1pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}