{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e3ada405-dcb3-50eb-8f65-9041052b8bea","task_key":"bbeh~5ftime~5farithmetic--full--e3ada405-dcb3-50eb-8f65-9041052b8bea","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: Rachel was born on 2012-Apr-05 and Brad was born on 2013-Jan-21. How many days is Rachel older than Brad? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: Sarah and Asher were born on 2021-Mar-16 and 2021-Jun-20 respectively. When Asher was 719 days old, how old was Sarah in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question3 be Z.\\n\\nQuestion3: Theodore was born on 2002-12-30. Compute his age in days on 2010-08-28. Both dates are in yyyy-mm-dd format. Your answer should be the number of days without the unit (for example, 233).\\n\\nLet X' = X - 288, Y' = Y - 788, and Z' = Z - 880.\\n\\nUse these values to solve Question4.\\n\\nQuestion4: If Bob, Lily, Emma, Jason, Dan have their exams at (date_times separated by ;): in September 2007; in January 1950; on X'-Y'-Z' at 03:39:35 PM; on 20 October, 1993 at 02:44:19 AM; in March 1918 respectively, arrange them in descending order. If a date/time is at a lower granularity, assume the earliest value for the missing information. Eg: if the date is Feb, 2020, assume the complete time to be 1 Feb 2020 00:00:00. \\\" Your answer should be an ordered list (for example, [answer1, answer2, ...]).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}