{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f83f43d8-38ed-59ab-9db1-8c16e31d544d","task_key":"bbeh~5ftime~5farithmetic--full--f83f43d8-38ed-59ab-9db1-8c16e31d544d","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: Asher was born on 1999-08-06. Compute his age in days on 2003-05-28. Both dates are in yyyy-mm-dd format. Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: If the current date in yyyy-mm-dd format is 2022-06-18 and I was born on 2020-10-27, what is my age in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet X' = X - 1371 and Y' = Y - 584.\\n\\nUse these values to solve Question3.\\n\\nQuestion3: A company pays their employees every X' days. Benjamin received his first paycheck on May Y', 2006. If today is Aug 05, 2006, when will he receive his next paycheck? Use the date format in the question to report the result.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}