{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"525b9e13-a120-59c7-b7b6-77121b74f2ab","task_key":"temporal~5fsequences--main--525b9e13-a120-59c7-b7b6-77121b74f2ab","task_revision_id":"1","upstream_id":"","short_description":"Today, Nancy went to the soccer field. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Nancy went to the soccer field. Between what times could they have gone?\\nWe know that:\\nNancy woke up at 5am.\\nWilliam saw Nancy attending class at the school from 5am to 11am.\\nLeslie saw Nancy waiting at the train station from 11am to 12pm.\\nNancy saw Nancy reading at the library from 3pm to 8pm.\\nThe soccer field was closed after 8pm.\\nBetween what times could Nancy have gone to the soccer field?\\nOptions:\\n(A) 3pm to 8pm\\n(B) 12pm to 3pm\\n(C) 5am to 11am\\n(D) 11am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}