{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"305f0124-9022-5012-8963-452e58acadd0","task_key":"temporal~5fsequences--main--305f0124-9022-5012-8963-452e58acadd0","task_revision_id":"1","upstream_id":"","short_description":"Today, David went to the gas station. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, David went to the gas station. Between what times could they have gone?\\nWe know that:\\nDavid woke up at 7am.\\nMichael saw David waiting at the train station from 7am to 12pm.\\nThomas saw David sitting on a rooftop from 12pm to 2pm.\\nTiffany saw David getting a coffee at the cafe from 2pm to 3pm.\\nJames saw David fixing their computer at the electronic store from 3pm to 5pm.\\nMark saw David reading at the library from 5pm to 6pm.\\nThe gas station was closed after 8pm.\\nBetween what times could David have gone to the gas station?\\nOptions:\\n(A) 2pm to 3pm\\n(B) 6pm to 8pm\\n(C) 5pm to 6pm\\n(D) 7am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}