{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0cf80839-fd1f-5857-bc2d-af7f144a88cb","task_key":"temporal~5fsequences--main--0cf80839-fd1f-5857-bc2d-af7f144a88cb","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the soccer field. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the soccer field. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 5am.\\nThomas saw Sarah sitting on a rooftop from 11am to 3pm.\\nAndrew saw Sarah fixing their computer at the electronic store from 3pm to 5pm.\\nEmily saw Sarah waiting at the airport from 5pm to 9pm.\\nThe soccer field was closed after 9pm.\\nBetween what times could Sarah have gone to the soccer field?\\nOptions:\\n(A) 3pm to 5pm\\n(B) 11am to 3pm\\n(C) 5am to 11am\\n(D) 5pm to 9pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}