{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4f733fa-ba16-529d-8329-95dfac0bdcbb","task_key":"temporal~5fsequences--main--e4f733fa-ba16-529d-8329-95dfac0bdcbb","task_revision_id":"1","upstream_id":"","short_description":"Today, Ashley went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Ashley went to the basketball court. Between what times could they have gone?\\nWe know that:\\nAshley woke up at 11am.\\nDavid saw Ashley buying a bike at the bike shop from 11am to 12pm.\\nWilliam saw Ashley sitting on a rooftop from 12pm to 3pm.\\nElizabeth saw Ashley working at the office from 3pm to 5pm.\\nThe basketball court was closed after 8pm.\\nBetween what times could Ashley have gone to the basketball court?\\nOptions:\\n(A) 3pm to 5pm\\n(B) 12pm to 3pm\\n(C) 5pm to 8pm\\n(D) 11am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}