{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14ae244b-fe3d-5062-a59c-6db0aaa63b1a","task_key":"bbeh~5fbuggy~5ftables--full--14ae244b-fe3d-5062-a59c-6db0aaa63b1a","task_revision_id":"1","upstream_id":"","short_description":"I have a table with 33 rows (including the header) and 18 columns. The table was…","config":"bbeh_buggy_tables","split":"full","body":"{\"input\":\"I have a table with 33 rows (including the header) and 18 columns. The table was saved in a column-major format as follows:\\n\\n[date, 2025-01-27, 2025-01-26, 2025-01-25, 2025-01-24, 2025-01-23, 2025-01-22, 2025-01-21, 2025-01-20, 2025-01-19, 2025-01-18, 2025-01-17, 2025-01-16, 2025-01-15, 2025-01-14, 2025-01-13, 2025-01-12, 2025-01-11, 2025-01-10, 2025-01-09, 2025-01-08, 2025-01-07, 2025-01-06, 2025-01-05, 2025-01-04, 2025-01-03, 2025-01-02, 2025-01-01, 2024-12-31, 2024-12-30, 2024-12-29, 2024-12-28, 2024-12-27, num_steps, null, 17239, 17600, 15886, 2139, 8631, 12397, 18625, 11212, 20000, 4314, null, 17985, 9880, 8425, 5489, 11525, 12729, 6864, 7371, 1312, 999, 19484, 12075, 3887, 3830, 4340, 8616, 9967, null, 4398, null, 9, study_minutes, 80, 79, 84, 77, 15, 49, 39, 65, 65, 22, 17, 98, 70, 75, 55, null, 40, 88, 99, 57, 88, 68, 87, 80, 50, 100, 85, null, null, 24, 78, 79, -3, -10, exercise_minutes, null, 93, 92, 87, 94, 20, 26, 79, null, 97, 54, 30, 20, 51, 63, 58, 34, 15, 24, 35, 13, 32, 21, 10, 61, 53, 96, 71, 10, 52, 69, 29, -7, -8, 0, sleep_minutes, 525, 364, 318, 403, 334, 533, 427, 536, 577, 571, null, 539, 531, 498, 340, 338, 520, 569, null, 397, 418, 577, 464, 550, 574, 588, 519, 514, 567, 451, 542, 324, -9, -7, -4, -5, bed_time, 22:11, 22:21, 21:54, 23:23, 23:38, 23:54, null, 22:26, 21:52, null, 23:39, 22:28, 23:58, 22:13, 23:18, 21:55, 22:12, 23:18, 23:59, 23:40, 21:15, 22:42, 23:11, 23:26, 23:42, 23:18, 23:43, 21:50, 21:26, 22:26, 23:48, 21:15, 9, 2, -10, 4, -5, num_messages, 24, 39, 27, 39, 71, 56, 63, null, 71, 36, 99, 54, 37, 61, 88, 49, 97, 87, 63, 40, 61, 79, 21, 98, 98, 18, 98, 26, 50, 17, 72, null, 9, 10, 8, 6, 8, 7, num_emails, 5, 1, null, 10, 9, 7, 4, 9, 1, 10, 1, 9, null, 2, null, null, null, 5, 5, 9, 10, 3, 5, 5, 2, 5, 10, 2, 7, 3, null, 3, -9, -7, 5, 5, -7, 4, -3, num_calls, null, 5, null, 8, 10, 5, 6, 1, 8, 8, null, 4, 3, 9, 5, 2, 6, 5, 8, 5, 8, null, 6, 1, null, null, 2, null, 2, 5, 4, null, 2, -1, 2, -3, -3, 7, 5, 6, calories_burned, 423, null, 849, 466, 146, 607, null, 692, 827, 909, 703, 881, 249, 742, 258, null, 460, 755, null, 607, 666, 798, 853, 181, 120, 888, null, 936, 991, 942, 367, 189, -4, -9, -9, 6, 8, -5, -5, -9, 0, weekday, null, Tue, null, Thu, Fri, Sat, Sun, null, Tue, Wed, Thu, null, Sat, null, Mon, Tue, Wed, Thu, Fri, Sat, Sun, Mon, Tue, Wed, Thu, Fri, Sat, Sun, null, null, Wed, Thu, -10, -4, -8, 9, 9, -3, 6, 3, 2, -7, calories_consumed, 399, 1326, 1238, 1078, 788, 971, null, 704, 773, 765, 1113, 1905, 613, 717, 1198, 1300, 1200, 541, 998, 431, 303, 444, 394, 1693, null, 1396, 1347, 1279, 524, 1208, 770, 610, -10, 5, 10, -9, 5, 9, 2, -5, 6, 3, -6, num_meetings, null, 5, 14, null, 8, 5, 11, 3, 5, 1, 6, 13, 13, 1, 10, 11, 6, 8, 11, 13, 2, 9, 5, 4, null, 8, 3, 6, 9, 14, 14, 14, 4, 5, -9, -10, -6, 2, 9, 3, -6, -7, -2, 1, coding_minutes, 18, 64, 33, 56, 23, 57, 35, 64, 29, 38, null, 60, 60, 16, 21, 21, 20, null, 16, 10, 17, 30, 2, 20, 30, 22, 24, 23, 52, null, 57, 33, 5, 7, 7, 1, 1, -8, 4, 1, -4, 1, -9, -4, -3, num_tasks_completed, 8, 3, 3, 2, 7, 7, 1, 5, 5, 4, 1, 3, 2, 5, 7, 2, null, 4, 2, 3, 7, 9, 10, 5, 4, 9, 7, null, 2, 8, 6, 5, -1, 4, 2, 6, 4, 7, -2, -1, -6, 4, -8, 3, 1, 3, water_intake_ml, 1673, 1904, 2244, 1505, 2154, 1179, 2667, 964, 2568, 1746, 2609, 1922, 2764, 624, 2875, 589, 1683, 938, 1225, 2206, null, 1529, 826, 707, 2200, 1173, 2286, null, 1042, 1319, 2017, 1915, 5, -2, 6, -7, 10, -7, -10, 8, 8, -7, 5, 7, 6, 8, 8, phone_screen_time_minutes, 37, 38, 51, 43, 23, 40, null, 32, 59, 19, 57, 35, 52, 35, 13, 31, 43, 30, 19, 56, 49, 39, 33, 20, 43, 60, 19, 52, 59, 14, 56, 19, -2, 6, 7, -6, -10, -8, -2, -3, -4, 8, -3, -7, -10, 4, -1, -8, music_listening_minutes, 36, null, 16, 49, 20, 12, 27, 55, 48, 55, 53, 29, 37, null, null, 28, 24, 57, 36, 15, 27, 11, 50, 13, 53, 29, 31, 18, 44, null, 13, 31, 5, 5, 6, -4, 5, -9, -1, 2, -4, 1, 9, 2, 5, -9, 9, -6, -3].\\n\\nHowever, the code to save the table was buggy and after saving each column, it appeneded some random values to the end of the column. Specifically, at the end of the i-th column (0-indexed), i random values were appended to the end of the column. Compute the absolute difference between the stdev of column phone_screen_time_minutes and column study_minutes, only looking at the days where the sleep minutes was greater than 350 and the number of tasks completed was greater than 3. Round the result to 2 decimal places. If it cannot be determined whether the condition is satisfied or not (e.g., due to null values that cannot be inferred from the rest of the table), assume the condition is not satisfied. If a value is null, ignore it from the computations. If any of the stdevs cannot be computed due to there being no values, the final answer must be \\\"0\\\". If there is only one value, the stdev must be 0. \"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}