{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdf8c0a5-e3be-5d6f-b77a-d60409bf798a","task_key":"bbeh~5ftime~5farithmetic--full--bdf8c0a5-e3be-5d6f-b77a-d60409bf798a","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X-Y-Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X-Y-Z.\\n\\nQuestion1: I was born on 2010-06-20 (yyyy-mm-dd) and my cousin was born 10 days before me. Give the date of birth of my cousin in the same format. If the question cannot be answered, reply \\\"unanswerable\\\". Use the date format in the question to answer the question.\\n\\nLet X' = X - 1998, Y' = Y + 18, and Z' = Z + 24.\\n\\nUse these values to solve Question2.\\n\\nQuestion2: David lives in Location A (UTC +0000) and is going to call James in Location B at X':Y':Z' AM, A's time. What time is it for B, who lives in PST (-0800), in their local time. Your answer should include the day and the time. The day should be the number of days before/after the start day (use same_day if the answer is the same day), and time should be in 24-hour format. Some examples are +2, 21:08:10 or same_day, 07:04.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}