{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4fd65419-2fef-5621-b5bb-7c1dca3d04eb","task_key":"bbeh~5ftime~5farithmetic--full--4fd65419-2fef-5621-b5bb-7c1dca3d04eb","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: If the current date is 2023-10-19 and I was born on 2018-04-26, what is my age in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: Abigail and Henry were born on 2016-May-30 and 2016-Nov-09 respectively. When Henry was 753 days old, how old was Abigail in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet X' = X - 1990 and Y' = Y - 896.\\n\\nUse these values to solve Question3.\\n\\nQuestion3: A company pays their employees every X' days. Benjamin received his first paycheck on Aug Y', 2021. If today is Jan Y', 2022, when will he receive his next paycheck? Use the date format in the question to report the result.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}