{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4cb837cc-2548-5d33-9de3-9d4753de4341","task_key":"temporal~5fsequences--main--4cb837cc-2548-5d33-9de3-9d4753de4341","task_revision_id":"1","upstream_id":"","short_description":"Today, Ashley went to the football field. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Ashley went to the football field. Between what times could they have gone?\\nWe know that:\\nAshley woke up at 1pm.\\nLisa saw Ashley taking photos near the Leaning Tower of Pisa from 2pm to 5pm.\\nNancy saw Ashley working out at the gym from 5pm to 6pm.\\nSteven saw Ashley working at the office from 6pm to 9pm.\\nKimberly saw Ashley buying a bike at the bike shop from 9pm to 10pm.\\nThe football field was closed after 10pm.\\nBetween what times could Ashley have gone to the football field?\\nOptions:\\n(A) 1pm to 2pm\\n(B) 5pm to 6pm\\n(C) 9pm to 10pm\\n(D) 2pm to 5pm\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}