{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6eb7449d-44c7-593f-bc42-63cea1f284fa","task_key":"temporal~5fsequences--main--6eb7449d-44c7-593f-bc42-63cea1f284fa","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the basketball court. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 6am.\\nThomas saw Jennifer buying a bike at the bike shop from 6am to 9am.\\nSamantha saw Jennifer driving to the water park from 9am to 10am.\\nSteven saw Jennifer fixing their computer at the electronic store from 10am to 1pm.\\nMark saw Jennifer stretching at a yoga studio from 3pm to 5pm.\\nThe basketball court was closed after 5pm.\\nBetween what times could Jennifer have gone to the basketball court?\\nOptions:\\n(A) 9am to 10am\\n(B) 6am to 9am\\n(C) 3pm to 5pm\\n(D) 1pm to 3pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}