{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ba33aa9-2386-5f4f-b575-739a0a887464","task_key":"bbeh~5ftime~5farithmetic--full--1ba33aa9-2386-5f4f-b575-739a0a887464","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X.\\n\\nQuestion1: Theodore was born on 1993-01-18. Compute his age in days on 2000-07-24. Both dates are in yyyy-mm-dd format. Your answer should be the number of days without the unit (for example, 233).\\n\\nLet the answer to Question2 be Y.\\n\\nQuestion2: Natalie and Chris were born on 2004-Feb-18 and 2004-Dec-30 respectively. When Chris was 991 days old, how old was Natalie in days? Your answer should be the number of days without the unit (for example, 233).\\n\\nLet X' = X - 2726 and Y' = Y - 1277.\\n\\nUse these values to solve Question3.\\n\\nQuestion3: Alan and Mary tried a new restaurant on  Nov Y', 2009 and really liked it. They decided to go to the same restaurant every X' days. If today is Feb 21, 2010, when is the next time they will go to that restaurant? Use the date format in the question to report the result.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}