{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a5af3fd-9f65-5554-9cbd-28bd596fc738","task_key":"temporal~5fsequences--main--8a5af3fd-9f65-5554-9cbd-28bd596fc738","task_revision_id":"1","upstream_id":"","short_description":"Today, Elizabeth went to the gas station. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Elizabeth went to the gas station. Between what times could they have gone?\\nWe know that:\\nElizabeth woke up at 5am.\\nMark saw Elizabeth buying a bike at the bike shop from 6am to 9am.\\nAndrew saw Elizabeth waiting at the airport from 9am to 10am.\\nSusan saw Elizabeth driving to the water park from 10am to 1pm.\\nLeslie saw Elizabeth waiting at the train station from 1pm to 2pm.\\nThomas saw Elizabeth attending class at the school from 2pm to 6pm.\\nThe gas station was closed after 6pm.\\nBetween what times could Elizabeth have gone to the gas station?\\nOptions:\\n(A) 5am to 6am\\n(B) 2pm to 6pm\\n(C) 1pm to 2pm\\n(D) 6am to 9am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}