{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40c2a810-e49b-52f8-9657-c17834927b52","task_key":"bbeh~5ftime~5farithmetic--full--40c2a810-e49b-52f8-9657-c17834927b52","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X-Y-Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X-Y-Z.\\n\\nQuestion1: What is the day 47 days from now in the yyyy-mm-dd format if the day before tomorrow is  2005-Oct-25? Your answer should be a date (for example, 2023-10-31).\\n\\nLet X' = X - 1999, Y' = Y - 2, and Z' = Z + 17.\\n\\nUse these values to solve Question2.\\n\\nQuestion2: Flight departs location A at 01:48 (24hr) UTC(+0000). It reaches location B at X':Y':Z' PM EST(-0500). What is the total time duration taken to fly? Your answer should be the number of days, hours, minutes, and seconds, respectively, without the units (for example, 3, 32, 46, 38).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}