{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72548891-933e-52be-8980-c2a809a7affa","task_key":"temporal~5fsequences--main--72548891-933e-52be-8980-c2a809a7affa","task_revision_id":"1","upstream_id":"","short_description":"Today, Ashley went to the beach. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Ashley went to the beach. Between what times could they have gone?\\nWe know that:\\nAshley woke up at 5am.\\nMark saw Ashley buying cookies at a bakery from 5am to 7am.\\nRichard saw Ashley sitting on a rooftop from 9am to 1pm.\\nWilliam saw Ashley walking in the garden from 1pm to 3pm.\\nHannah saw Ashley taking photos near the Leaning Tower of Pisa from 3pm to 8pm.\\nSarah saw Ashley getting a coffee at the cafe from 8pm to 9pm.\\nThe beach was closed after 9pm.\\nBetween what times could Ashley have gone to the beach?\\nOptions:\\n(A) 5am to 7am\\n(B) 3pm to 8pm\\n(C) 9am to 1pm\\n(D) 7am to 9am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}