{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3fbd8dd1-0230-523a-8e1c-15255952f688","task_key":"temporal~5fsequences--main--3fbd8dd1-0230-523a-8e1c-15255952f688","task_revision_id":"1","upstream_id":"","short_description":"Today, Hannah went to the bakery. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Hannah went to the bakery. Between what times could they have gone?\\nWe know that:\\nHannah woke up at 5am.\\nHannah saw Hannah buying a phone at the electronics store from 11am to 2pm.\\nAshley saw Hannah taking photos near the Eiffel Tower from 2pm to 3pm.\\nNancy saw Hannah buying clothes at the mall from 3pm to 5pm.\\nThe bakery was closed after 5pm.\\nBetween what times could Hannah have gone to the bakery?\\nOptions:\\n(A) 2pm to 3pm\\n(B) 5am to 11am\\n(C) 3pm to 5pm\\n(D) 11am to 2pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}