{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79923470-2a3f-5b38-b7d9-ffcedef41c34","task_key":"bbeh~5ftime~5farithmetic--full--79923470-2a3f-5b38-b7d9-ffcedef41c34","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: Clara and William were born on 2015-Aug-24 and 2016-May-20 respectively. When William was 326 days old, how old was Clara in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: The war had been going on for 970 days on 2015-08-30. How many days had it been going on for on 2015-09-29, if it never stopped in between the 2 dates? (Answer in days). Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question3 be Z.\\n\\nQuestion3: Today is 2007-09-28 and I was born on 2005-12-28. I am M months old. Compute M. You can assume the time between the same date in two consecutive months to be 1 month. Eg: the time between 02/13/2020 and 03/13/2020 is 1 month. Your answer should be the number of months without the unit (for example, 12).\\n\\nLet X' = X - 585, Y' = Y - 956, and Z' = Z + 2.\\n\\nUse these values to solve Question4.\\n\\nQuestion4: If Lily, Emma, Bob, Joe, Jenny have their exams at (times separated by ;):  01 PM; 01:50 AM; X':Y':Z' (24hr); 18:58 (24hr); 06:35:30 PM respectively, find the persons whose exams are strictly after the cut-off time of 07:02:35 (24hr). If a time is at a lower granularity, assume the earliest value for the missing information. Eg: if the time is 2:40 PM, assume the complete time to be 2:40:00 PM. Your answer should be an ordered list with alphabetically ordered items (for example, [answer1, answer2, ...]). If there are no valid answers, return an empty list [].\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}