{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd18dbb8-9643-5c34-990d-68ab02e217ab","task_key":"temporal~5fsequences--main--dd18dbb8-9643-5c34-990d-68ab02e217ab","task_revision_id":"1","upstream_id":"","short_description":"Today, Ashley went to the bookstore. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Ashley went to the bookstore. Between what times could they have gone?\\nWe know that:\\nAshley woke up at 5am.\\nJason saw Ashley buying lunch at the deli from 11am to 1pm.\\nEmily saw Ashley driving to the water park from 1pm to 3pm.\\nKimberly saw Ashley stretching at a yoga studio from 3pm to 6pm.\\nThe bookstore was closed after 6pm.\\nBetween what times could Ashley have gone to the bookstore?\\nOptions:\\n(A) 1pm to 3pm\\n(B) 11am to 1pm\\n(C) 3pm to 6pm\\n(D) 5am to 11am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}