{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e4f75fa-c0f0-5fa5-ad8f-2183ebb6d680","task_key":"temporal~5fsequences--main--5e4f75fa-c0f0-5fa5-ad8f-2183ebb6d680","task_revision_id":"1","upstream_id":"","short_description":"Today, Emily went to the soccer field. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Emily went to the soccer field. Between what times could they have gone?\\nWe know that:\\nEmily woke up at 10am.\\nLinda saw Emily waiting at the train station from 10am to 12pm.\\nAshley saw Emily fixing their computer at the electronic store from 12pm to 1pm.\\nJohn saw Emily walking towards the Statue of Liberty from 1pm to 2pm.\\nThe soccer field was closed after 8pm.\\nBetween what times could Emily have gone to the soccer field?\\nOptions:\\n(A) 12pm to 1pm\\n(B) 10am to 12pm\\n(C) 2pm to 8pm\\n(D) 1pm to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}