{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5bbb339-da0b-5b35-9a9a-c64c61eebba4","task_key":"temporal~5fsequences--main--b5bbb339-da0b-5b35-9a9a-c64c61eebba4","task_revision_id":"1","upstream_id":"","short_description":"Today, William went to the soccer field. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, William went to the soccer field. Between what times could they have gone?\\nWe know that:\\nWilliam woke up at 1pm.\\nSarah saw William working at the office from 1pm to 2pm.\\nMary saw William waiting at the train station from 2pm to 4pm.\\nSusan saw William taking photos near the Eiffel Tower from 8pm to 10pm.\\nThe soccer field was closed after 10pm.\\nBetween what times could William have gone to the soccer field?\\nOptions:\\n(A) 8pm to 10pm\\n(B) 1pm to 2pm\\n(C) 2pm to 4pm\\n(D) 4pm to 8pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}