{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89aa5850-2ced-539e-8b42-44168a709c68","task_key":"temporal~5fsequences--main--89aa5850-2ced-539e-8b42-44168a709c68","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the basketball court. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 7am.\\nLisa saw William stretching at a yoga studio from 11am to 2pm.\\nJohn saw William walking towards the Statue of Liberty from 2pm to 3pm.\\nSean saw William reading at the library from 3pm to 5pm.\\nSarah saw William driving to the water park from 5pm to 9pm.\\nThe basketball court was closed after 9pm.\\nBetween what times could William have gone to the basketball court?\\nOptions:\\n(A) 5pm to 9pm\\n(B) 3pm to 5pm\\n(C) 7am to 11am\\n(D) 11am to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}