{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d50fdd01-005c-5e47-b2f0-7693e47de87a","task_key":"temporal~5fsequences--main--d50fdd01-005c-5e47-b2f0-7693e47de87a","task_revision_id":"1","upstream_id":"","short_description":"Today, Sean went to the soccer field. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sean went to the soccer field. Between what times could they have gone?\\nWe know that:\\nSean woke up at 5am.\\nMary saw Sean walking towards the Statue of Liberty from 7am to 1pm.\\nAnthony saw Sean fixing their computer at the electronic store from 1pm to 4pm.\\nWilliam saw Sean watching a movie at the theater from 4pm to 5pm.\\nSarah saw Sean getting a coffee at the cafe from 5pm to 8pm.\\nThe soccer field was closed after 8pm.\\nBetween what times could Sean have gone to the soccer field?\\nOptions:\\n(A) 1pm to 4pm\\n(B) 4pm to 5pm\\n(C) 5pm to 8pm\\n(D) 5am to 7am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}