{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b93453cd-072c-504d-a424-fef99789fe71","task_key":"bbeh~5ftime~5farithmetic--full--b93453cd-072c-504d-a424-fef99789fe71","task_revision_id":"1","upstream_id":"","short_description":"Let the answer to Question1 be X, Y, Z.","config":"bbeh_time_arithmetic","split":"full","body":"{\"input\":\"Let the answer to Question1 be X, Y, Z.\\n\\nQuestion1: It takes Hannah an average of 13 minutes and 46 seconds to study 2 pages. If she wants to study 16 pages at the same rate, it will take her A hours, B minutes, and C seconds. Your answer should be the value of A, B, and C, respectively (for example, 2, 5, 10).\\n\\nLet X' = X + 2, Y' = Y + 9, and Z' = Z + 41.\\n\\nLet the answer to Question2 be P, R, Q.\\n\\nQuestion2: Alice has bought a robot that helps with everyday tasks. It takes the robot X':Y':Z' hours:minutes:seconds to respond to an email, on average. Alice wants the robot to respond to 7 emails. It took the robot A hours, B minutes, and C seconds. Your answer should be the value of A, B, and C, respectively (for example, 22, 58, 10).\\n\\nLet P' = P + 1974, R' = R - 56, and Q' = Q - 12.\\n\\nUse these values to answer Question3.\\n\\nQuestion3: The most important game was on P'-R'-Q' (yyyy-mm-dd) and the second most important game was 6 days before it. What was the date of the second most important game? If the question cannot be answered, reply \\\"unanswerable\\\". Use the date format in the question to answer the question.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}