{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79bc97be-18b4-5879-ab1f-d1d8b6b8abc1","task_key":"temporal~5fsequences--main--79bc97be-18b4-5879-ab1f-d1d8b6b8abc1","task_revision_id":"1","upstream_id":"","short_description":"Today, Jessica went to the beach. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jessica went to the beach. Between what times could they have gone?\\nWe know that:\\nJessica woke up at 5am.\\nEmily saw Jessica stretching at a yoga studio from 5am to 8am.\\nThomas saw Jessica taking photos near the Eiffel Tower from 8am to 9am.\\nRichard saw Jessica fixing their computer at the electronic store from 12pm to 3pm.\\nKimberly saw Jessica watching a movie at the theater from 3pm to 7pm.\\nSarah saw Jessica driving to the water park from 7pm to 10pm.\\nThe beach was closed after 10pm.\\nBetween what times could Jessica have gone to the beach?\\nOptions:\\n(A) 8am to 9am\\n(B) 12pm to 3pm\\n(C) 9am to 12pm\\n(D) 5am to 8am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}