{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"099217c7-9c9e-527f-b0d5-9b7b731466c4","task_key":"temporal~5fsequences--main--099217c7-9c9e-527f-b0d5-9b7b731466c4","task_revision_id":"1","upstream_id":"","short_description":"Today, Ashley went to the basketball court. Between what times could they have…","config":"temporal_sequences","split":"main","body":"{\"input\":\"Today, Ashley went to the basketball court. Between what times could they have gone?\\nWe know that:\\nAshley woke up at 7am.\\nLisa saw Ashley working at the office from 7am to 8am.\\nMichael saw Ashley walking in the garden from 8am to 10am.\\nSamantha saw Ashley attending class at the school from 1pm to 6pm.\\nThe basketball court was closed after 6pm.\\nBetween what times could Ashley have gone to the basketball court?\\nOptions:\\n(A) 8am to 10am\\n(B) 1pm to 6pm\\n(C) 10am to 1pm\\n(D) 7am to 8am\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}