{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1e933e8-3f52-5b5a-bbce-9d000ad320d1","task_key":"temporal~5fsequences--main--e1e933e8-3f52-5b5a-bbce-9d000ad320d1","task_revision_id":"1","upstream_id":"","short_description":"Today, Sean went to the amusement park. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sean went to the amusement park. Between what times could they have gone?\\nWe know that:\\nSean woke up at 11am.\\nLeslie saw Sean waiting at the train station from 11am to 1pm.\\nWilliam saw Sean buying clothes at the mall from 1pm to 3pm.\\nJennifer saw Sean sitting on a rooftop from 6pm to 8pm.\\nThe amusement park was closed after 8pm.\\nBetween what times could Sean have gone to the amusement park?\\nOptions:\\n(A) 6pm to 8pm\\n(B) 1pm to 3pm\\n(C) 11am to 1pm\\n(D) 3pm to 6pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}