{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa867173-6e9e-5281-8d6b-ef4f8325f57f","task_key":"bbeh~5ftime~5farithmetic--full--aa867173-6e9e-5281-8d6b-ef4f8325f57f","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X, Y, Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X, Y, Z.\\n\\nQuestion1: Amelia has created a robot to do her tasks. It takes the robot 3 hours, 52 minutes, and 54 seconds to traslate one page, on average. If the robot is tasked to traslate 23 pages, it will take it A hours, B minutes, and C seconds. Your answer should be the value of A, B, and C, respectively (for example, 22, 58, 10).\\n\\nLet X' = X - 88, Y' = Y - 6, and Z' = Z + 8.\\n\\nLet the answer to Question2 be P, R, Q.\\n\\nQuestion2: Three students measured the time it took them to complete an assignment. It took the first student X':Y':Z', the second student 1:00:40, and the third 1:30:00 hours:minutes:seconds to finish their assignment. They want to compute the average time H:M:S. Help them by computing  H, M and S and reporting. Your answer should be the hours, minutes, and seconds, respectively, without the units (for example, 1, 58, 10).\\n\\nLet P' = P + 1999, R' = R - 4, and Q' = Q - 31.\\n\\nUse these values to answer Question3.\\n\\nQuestion3: I was born on P'-R'-Q' (yyyy-mm-dd) and my cousin was born 10 days before me. Give the date of birth of my cousin in the same format. If the question cannot be answered, reply \\\"unanswerable\\\". Use the date format in the question to answer the question.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}