{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"213f987f-aa52-502a-964e-63b71ea46e63","task_key":"bbeh~5ftime~5farithmetic--full--213f987f-aa52-502a-964e-63b71ea46e63","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: Joe was born on 2008-12-07. Compute his age in days on 2010-04-20. Both dates are in yyyy-mm-dd format. Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: Theodore was born on 2013-02-25. Compute his age in days on 2019-02-21. Both dates are in yyyy-mm-dd format. Your answer should be the number of days without the unit (for example, 233).\\n\\nLet X' = X - 480 and Y' = Y - 2157.\\n\\nUse these values to solve Question3.\\n\\nQuestion3: A museum announced that every X' days, there will be a day of free entrance. The first free day was on Y' September, 2004. If today is Y' November, 2004, when is the next free day? Use the date format in the question to report the result.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}