{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"113f7fe0-3b32-57b2-a5ae-024d3e3ad4ad","task_key":"temporal~5fsequences--main--113f7fe0-3b32-57b2-a5ae-024d3e3ad4ad","task_revision_id":"1","upstream_id":"","short_description":"Today, Sarah went to the gas station. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Sarah went to the gas station. Between what times could they have gone?\\nWe know that:\\nSarah woke up at 6am.\\nSamantha saw Sarah reading at the library from 6am to 9am.\\nMary saw Sarah taking photos near the Eiffel Tower from 9am to 11am.\\nSteven saw Sarah buying cookies at a bakery from 11am to 2pm.\\nThe gas station was closed after 9pm.\\nBetween what times could Sarah have gone to the gas station?\\nOptions:\\n(A) 11am to 2pm\\n(B) 2pm to 9pm\\n(C) 9am to 11am\\n(D) 6am to 9am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}