{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65e8fd5f-2f41-58ca-b69e-a3e46d050bc9","task_key":"bbeh~5ftime~5farithmetic--full--65e8fd5f-2f41-58ca-b69e-a3e46d050bc9","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: Natalie was born on 2014-Aug-17 and Alex was born on 2017-May-17. Alex is M months younger than Natalie. Compute M.  You can assume the time between the same date in two consecutive months to be 1 month. Eg: the time between 02/13/2020 and 03/13/2020 is 1 month. Your answer should be the number of months without the unit (for example, 12).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: Today is 2014-07-27 (yyyy-mm-dd format) and I was born on 2012-05-27. I am M months old. Compute M. You can assume the time between the same date in two consecutive months to be 1 month. Eg: the time between 02/13/2020 and 03/13/2020 is 1 month. Your answer should be the number of months without the unit (for example, 12).\\n\\nLet X' = X - 17 and Y' = Y + 2.\\n\\nUse these values to solve Question3.\\n\\nQuestion3: Alan and Mary tried a new restaurant on  May Y', 2004 and really liked it. They decided to go to the same restaurant every X' days. If today is Dec 07, 2004, when is the next time they will go to that restaurant? Use the date format in the question to report the result.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}