{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c548321-244c-51dd-8091-3671135e44f9","task_key":"temporal~5fsequences--main--2c548321-244c-51dd-8091-3671135e44f9","task_revision_id":"1","upstream_id":"","short_description":"Today, James went to the dance studio. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, James went to the dance studio. Between what times could they have gone?\\nWe know that:\\nJames woke up at 7am.\\nThomas saw James taking photos near the Eiffel Tower from 7am to 12pm.\\nMark saw James driving to the water park from 12pm to 2pm.\\nAnthony saw James buying a phone at the electronics store from 2pm to 4pm.\\nSarah saw James buying lunch at the deli from 5pm to 6pm.\\nThe dance studio was closed after 6pm.\\nBetween what times could James have gone to the dance studio?\\nOptions:\\n(A) 2pm to 4pm\\n(B) 7am to 12pm\\n(C) 4pm to 5pm\\n(D) 12pm to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}