{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db7a4713-d0b2-5ed6-8a19-755260cca4c8","task_key":"temporal~5fsequences--main--db7a4713-d0b2-5ed6-8a19-755260cca4c8","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the amusement park. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the amusement park. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 5am.\\nLisa saw Jennifer buying a bike at the bike shop from 5am to 10am.\\nSamantha saw Jennifer taking photos near the Eiffel Tower from 10am to 2pm.\\nWilliam saw Jennifer waiting at the airport from 2pm to 3pm.\\nAshley saw Jennifer buying lunch at the deli from 4pm to 5pm.\\nAndrew saw Jennifer working at the office from 5pm to 8pm.\\nThe amusement park was closed after 8pm.\\nBetween what times could Jennifer have gone to the amusement park?\\nOptions:\\n(A) 2pm to 3pm\\n(B) 5pm to 8pm\\n(C) 5am to 10am\\n(D) 3pm to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}