{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2466abb-8ea1-5bf9-ab0e-decfac101cd1","task_key":"temporal~5fsequences--main--b2466abb-8ea1-5bf9-ab0e-decfac101cd1","task_revision_id":"1","upstream_id":"","short_description":"Today, Ashley went to the beach. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Ashley went to the beach. Between what times could they have gone?\\nWe know that:\\nAshley woke up at 7am.\\nElizabeth saw Ashley stretching at a yoga studio from 7am to 8am.\\nSamantha saw Ashley buying lunch at the deli from 8am to 11am.\\nJennifer saw Ashley buying clothes at the mall from 11am to 5pm.\\nThe beach was closed after 6pm.\\nBetween what times could Ashley have gone to the beach?\\nOptions:\\n(A) 8am to 11am\\n(B) 11am to 5pm\\n(C) 5pm to 6pm\\n(D) 7am to 8am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}