{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"456c9498-309f-5d24-9e21-1f63e8c30c77","task_key":"bbeh~5ftime~5farithmetic--full--456c9498-309f-5d24-9e21-1f63e8c30c77","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: I was exactly 585 days old on 2018-10-27. Compute my age in days on 2019-09-24. Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: If the current date in yyyy-mm-dd format is 2018-12-27 and I was born on 2012-01-14, what is my age in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question3 be Z.\\n\\nQuestion3: In a fantasy movie, if the mythological character's birthday is on 14 December, 1873, how many years old is he on 16 May, 1904? If the information provided is not enough to calculate the answer, or if there is more than one possible answer, reply with unanswerable.\\n\\nLet X' = X + 1048, Y' = Y - 2532, and Z' = Z - 12.\\n\\nUse these values to solve Question4.\\n\\nQuestion4: In a movie, here are some activities David did. He was camping in June 1758. He also did some gardening and snowboarding on 05-13-2037 and in October 1874 respectively. He was also hunting on May 13, 2037 at 01 PM and skateboarding on May 19, 1842. Find the activities which are strictly after the cut off date/time of X'-Y'-Z' 13:09:19 (24hr). If a date/time is at a lower granularity, assume the earliest value for the missing information. Eg: if the date is Feb, 2020, assume the complete date/time to be 1 Feb 2020 00:00:00. Your answer should be an ordered list with alphabetically ordered items (for example, [cooking, playing, ...]). If there are no valid answers, return an empty list [].\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}