{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9be0214-cceb-5961-be21-ac876c5fa8f4","task_key":"temporal~5fsequences--main--f9be0214-cceb-5961-be21-ac876c5fa8f4","task_revision_id":"1","upstream_id":"","short_description":"Today, Jessica went to the restaurant. Between what times could they have gone?","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jessica went to the restaurant. Between what times could they have gone?\\nWe know that:\\nJessica woke up at 9am.\\nNancy saw Jessica attending class at the school from 9am to 11am.\\nJohn saw Jessica taking photos near the Leaning Tower of Pisa from 11am to 12pm.\\nMary saw Jessica taking photos near the Eiffel Tower from 12pm to 1pm.\\nAnthony saw Jessica buying a phone at the electronics store from 1pm to 4pm.\\nKimberly saw Jessica buying cookies at a bakery from 4pm to 5pm.\\nThe restaurant was closed after 6pm.\\nBetween what times could Jessica have gone to the restaurant?\\nOptions:\\n(A) 9am to 11am\\n(B) 4pm to 5pm\\n(C) 5pm to 6pm\\n(D) 11am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}