{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1eb7142-ac17-5679-8b3d-254c984166cd","task_key":"bbeh~5ftime~5farithmetic--full--e1eb7142-ac17-5679-8b3d-254c984166cd","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X-Y-Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X-Y-Z.\\n\\nQuestion1: I was born on 2006-06-13 (yyyy-mm-dd) and my cousin was born 5 days before me. Give the date of birth of my cousin in the same format. If the question cannot be answered, reply \\\"unanswerable\\\". Use the date format in the question to answer the question.\\n\\nLet Y' = Y + 4 and Z' = Z - 2.\\n\\nUse these values to solve Question2.\\n\\nQuestion2: If right now, it is Y':Z' PM in Location A (UTC +0000), what time is it in Location B (EST -0500)? Your answer should include the day and the time. The day should be the number of days before/after the start day (use same_day if the answer is the same day), and time should be in 24-hour format. Some examples are +2, 21:44:10 or same_day, 11:04.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}