{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29a9fb51-f80b-5404-8906-6fc93e340841","task_key":"temporal~5fsequences--main--29a9fb51-f80b-5404-8906-6fc93e340841","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the bakery. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the bakery. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 9am.\\nJohn saw Sarah getting a coffee at the cafe from 10am to 12pm.\\nThomas saw Sarah buying lunch at the deli from 12pm to 4pm.\\nRichard saw Sarah driving to the water park from 4pm to 5pm.\\nThe bakery was closed after 5pm.\\nBetween what times could Sarah have gone to the bakery?\\nOptions:\\n(A) 4pm to 5pm\\n(B) 12pm to 4pm\\n(C) 9am to 10am\\n(D) 10am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}