{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f4db7ae-f952-5ed6-8b55-5fa53b7fefe7","task_key":"bbeh~5ftime~5farithmetic--full--6f4db7ae-f952-5ed6-8b55-5fa53b7fefe7","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: The war had been going on for 766 days on 2010-03-27. How many days had it been going on for on 2010-09-07, if it never stopped in between the 2 dates? (Answer in days). Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: In a fantasy movie, if the mythological character's birthday is on 20 January, 1947, how many years old is he on 24 June, 1999? If the information provided is not enough to calculate the answer, or if there is more than one possible answer, reply with unanswerable.\\n\\nLet the answer to Question3 be Z.\\n\\nQuestion3: Emma and Levi were born on 2003-Oct-17 and 2003-Dec-24 respectively. When Levi was 840 days old, how old was Emma in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet X' = X - 923, Y' = Y - 12, and Z' = Z - 891.\\n\\nUse these values to solve Question4.\\n\\nQuestion4: If Joe, Lily, Emma, Jason, Dan have their exams at (times separated by ;):  X':Y':Z' (24hr); 20:05 (24hr); 02:21:02 (24hr); 11:14 PM; 20:01 (24hr) respectively, arrange the persons in descending order of their exam time. If a time is at a lower granularity, assume the earliest value for the missing information. Eg: if the time is 2:40 PM, assume the complete time to be 2:40:00 PM. Your answer should be an ordered list (for example, [answer1, answer2, ...]).\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}