{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96ec109f-046d-5b10-8c3c-e36963687933","task_key":"temporal~5fsequences--main--96ec109f-046d-5b10-8c3c-e36963687933","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the dance studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the dance studio. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 5am.\\nJason saw Sarah watching a movie at the theater from 5am to 8am.\\nHannah saw Sarah sitting on a rooftop from 10am to 3pm.\\nAndrew saw Sarah waiting at the train station from 3pm to 9pm.\\nThe dance studio was closed after 9pm.\\nBetween what times could Sarah have gone to the dance studio?\\nOptions:\\n(A) 3pm to 9pm\\n(B) 8am to 10am\\n(C) 10am to 3pm\\n(D) 5am to 8am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}