{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"07d3dcb3-106e-5ed2-aa37-0bf13c1e6be9","task_key":"temporal~5fsequences--main--07d3dcb3-106e-5ed2-aa37-0bf13c1e6be9","task_revision_id":"1","upstream_id":"","short_description":"Today, Tiffany went to the construction site. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Tiffany went to the construction site. Between what times could they have gone?\\nWe know that:\\nTiffany woke up at 7am.\\nWilliam saw Tiffany attending class at the school from 7am to 9am.\\nSteven saw Tiffany waiting at the airport from 9am to 1pm.\\nMark saw Tiffany buying lunch at the deli from 4pm to 7pm.\\nThe construction site was closed after 7pm.\\nBetween what times could Tiffany have gone to the construction site?\\nOptions:\\n(A) 4pm to 7pm\\n(B) 9am to 1pm\\n(C) 7am to 9am\\n(D) 1pm to 4pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}