{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"889b7ef7-af0a-56de-a71e-58eb48f727e7","task_key":"temporal~5fsequences--main--889b7ef7-af0a-56de-a71e-58eb48f727e7","task_revision_id":"1","upstream_id":"","short_description":"Today, Samantha went to the bakery. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Samantha went to the bakery. Between what times could they have gone?\\nWe know that:\\nSamantha woke up at 6am.\\nSarah saw Samantha walking in the garden from 6am to 11am.\\nSean saw Samantha taking photos near the Leaning Tower of Pisa from 12pm to 1pm.\\nJason saw Samantha taking photos near the Eiffel Tower from 1pm to 2pm.\\nElizabeth saw Samantha walking towards the Statue of Liberty from 2pm to 3pm.\\nTiffany saw Samantha buying lunch at the deli from 3pm to 9pm.\\nThe bakery was closed after 9pm.\\nBetween what times could Samantha have gone to the bakery?\\nOptions:\\n(A) 1pm to 2pm\\n(B) 11am to 12pm\\n(C) 3pm to 9pm\\n(D) 6am to 11am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}