{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7174d0d7-a2e4-5bbd-8bdf-35d9a5bef192","task_key":"temporal~5fsequences--main--7174d0d7-a2e4-5bbd-8bdf-35d9a5bef192","task_revision_id":"1","upstream_id":"","short_description":"Today, Elizabeth went to the movies. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Elizabeth went to the movies. Between what times could they have gone?\\nWe know that:\\nElizabeth woke up at 5am.\\nSarah saw Elizabeth getting a coffee at the cafe from 5am to 3pm.\\nSusan saw Elizabeth working out at the gym from 3pm to 4pm.\\nJohn saw Elizabeth buying a phone at the electronics store from 4pm to 6pm.\\nTiffany saw Elizabeth waiting at the train station from 6pm to 7pm.\\nAndrew saw Elizabeth waiting at the airport from 7pm to 8pm.\\nThe movies was closed after 9pm.\\nBetween what times could Elizabeth have gone to the movies?\\nOptions:\\n(A) 8pm to 9pm\\n(B) 4pm to 6pm\\n(C) 5am to 3pm\\n(D) 6pm to 7pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}