{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb889731-d70a-5923-849d-75e47ba7e2c6","task_key":"bbeh~5ftime~5farithmetic--full--cb889731-d70a-5923-849d-75e47ba7e2c6","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X-Y-Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X-Y-Z.\\n\\nQuestion1: If the date for the day before tomorrow in yyyy-mm-dd format is 2011-02-28, what is the date 85 days from now in the same format? Your answer should be a date (for example, 2023-10-31).\\n\\nLet Y' = Y + 1 and Z' = Z - 0.\\n\\nUse these values to solve Question2.\\n\\nQuestion2: Flight departs location A at 01 PM EST(-0500). It reaches location B at Y':Z' PM UTC(+0000). What is the total time duration taken to fly? Your answer should be the number of days, hours, minutes, and seconds, respectively, without the units (for example, 3, 32, 46, 38).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}