{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"372b3959-47d9-594a-9785-1feb8a8fc05c","task_key":"bbeh~5ftime~5farithmetic--full--372b3959-47d9-594a-9785-1feb8a8fc05c","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X, Y, Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X, Y, Z.\\n\\nQuestion1: The average time taken to practice a song is 28 minutes and 20 seconds. Calculate the time taken to practice 21 songs at the same rate. Your answer should be the value of X, Y, and Z, respectively (for example, 2, 5, 10).\\n\\nLet X' = X - 6, Y' = Y - 22, and Z' = Z + 29.\\n\\nLet the answer to Question2 be P, R, Q.\\n\\nQuestion2: Amelia has bought a robot that helps with everyday tasks. It takes the robot X':Y':Z' hours:minutes:seconds to read a chapter, on average. Amelia wants the robot to read 16 chapters. It took the robot A hours, B minutes, and C seconds. Your answer should be the value of A, B, and C, respectively (for example, 22, 58, 10).\\n\\nLet P' = P + 1948, R' = R - 53, and Q' = Q - 12.\\n\\nUse these values to answer Question3.\\n\\nQuestion3: If the date for today in yyyy-mm-dd format is P'-R'-Q', what was the date 5 days ago in the same format? If the question cannot be answered, reply \\\"unanswerable\\\". Use the date format in the question to answer the question.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}