{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13805a65-6455-5b35-a9cf-40c8af1a94d0","task_key":"temporal~5fsequences--main--13805a65-6455-5b35-a9cf-40c8af1a94d0","task_revision_id":"1","upstream_id":"","short_description":"Today, Jennifer went to the orchestra hall. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Jennifer went to the orchestra hall. Between what times could they have gone?\\nWe know that:\\nJennifer woke up at 5am.\\nSusan saw Jennifer buying a bike at the bike shop from 5am to 12pm.\\nJessica saw Jennifer working at the office from 12pm to 4pm.\\nNancy saw Jennifer buying clothes at the mall from 5pm to 9pm.\\nThe orchestra hall was closed after 9pm.\\nBetween what times could Jennifer have gone to the orchestra hall?\\nOptions:\\n(A) 12pm to 4pm\\n(B) 5pm to 9pm\\n(C) 4pm to 5pm\\n(D) 5am to 12pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}