{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e137301-2334-528f-9b5a-8a4bf3dec334","task_key":"bbeh~5fbuggy~5ftables--full--7e137301-2334-528f-9b5a-8a4bf3dec334","task_revision_id":"1","upstream_id":"","short_description":"I have a table with 31 rows (including the header) and 18 columns. The table was…","config":"bbeh_buggy_tables","split":"full","body":"{\"input\":\"I have a table with 31 rows (including the header) and 18 columns. The table was converted to markdown format as follows:\\n\\n| date |num_steps|study_minutes|exercise_minutes|sleep_minutes|bed_time|num_messages|num_emails|num_calls|calories_burned|weekday|calories_consumed|num_meetings|coding_minutes|num_tasks_completed|water_intake_ml|phone_screen_time_minutes|music_listening_minutes |\\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\\n| 12   | 2025-01-27| 12409| 41   | 13   | 334  | 23:56| 66   | 5    | 6    | 130  | Mon  | 1855 | 1    | 56   | 5    | 1646 | 55    |\\n| 34   | null | 2025-01-26| 6968 | 21   | 23   | 486  | 23:26| 89   | 10   | null | 806  | Tue  | 1135 | 3    | 45   | 9    | 695   |\\n| 1218 | null | null | 2025-01-25| null | 52   | 30   | 555  | 22:15| 86   | 5    | 6    | 487  | Wed  | 873  | 15   | 45   | 2     |\\n| null | 1306 | 17   | 22   | 2025-01-24| 15472| 79   | 52   | 512  | 21:23| 63   | 5    | 5    | 290  | Thu  | 1265 | 3    | 33    |\\n| null | 10   | 887  | 37   | 52   | 2025-01-23| 10736| 55   | 19   | 556  | 23:22| 26   | 7    | 4    | 465  | Fri  | 463  | 3     |\\n| 2    | 43   | 9    | 1192 | 39   | 40   | 2025-01-22| 18231| 31   | 65   | 370  | 23:47| 10   | 5    | 8    | 540  | Sat  | 1829  |\\n| 352  | 12   | 41   | 9    | 2560 | 56   | 35   | 2025-01-21| 19968| 69   | 79   | 440  | 21:56| 85   | 7    | 1    | 927  | Sun   |\\n| Mon  | 1609 | 4    | 42   | 10   | 2820 | 46   | 45   | 2025-01-20| 14914| null | 37   | 493  | 21:36| 26   | 5    | 7    | 952   |\\n| 725  | Tue  | null | 15   | 17   | 7    | 502  | 38   | 14   | 2025-01-19| 12300| 91   | 62   | 581  | 22:41| null | 4    | 10    |\\n| 7    | 593  | Wed  | 318  | 6    | null | 9    | 1321 | 43   | 30   | 2025-01-18| 965  | null | 15   | 469  | 22:10| 18   | 2     |\\n| 6    | 3    | 487  | Thu  | 1859 | 13   | 26   | 10   | 2052 | 18   | 44   | 2025-01-17| 6435 | 52   | null | 335  | 21:12| 83    |\\n| null | 4    | null | 176  | Fri  | 784  | 6    | 48   | 9    | 1492 | 59   | 33   | 2025-01-16| 4066 | null | 33   | null | 21:42 |\\n| 21:17| null | 6    | 4    | 533  | null | 799  | 11   | 12   | 5    | 1223 | 10   | null | 2025-01-15| 4177 | 89   | 92   | 429   |\\n| 598  | 23:39| 100  | 9    | 10   | null | Sun  | 722  | 2    | 44   | 2    | 1707 | 52   | 16   | 2025-01-14| 139  | 19   | 31    |\\n| 97   | 494  | 21:44| 100  | 7    | 4    | 863  | Mon  | null | 9    | 57   | 5    | 2939 | 54   | 29   | 2025-01-13| 10009| 14    |\\n| 73   | null | 490  | 21:46| 88   | 2    | 1    | 310  | Tue  | 1687 | 5    | 26   | 10   | null | 53   | 36   | 2025-01-12| 3791  |\\n| 8426 | null | 72   | 445  | 23:44| 93   | null | 1    | 888  | Wed  | 413  | 13   | 48   | null | 1143 | 59   | 40   | 2025-01-11 |\\n| 2025-01-10| 18217| 75   | 57   | 408  | 21:56| 90   | null | 5    | 797  | Thu  | 1922 | 1    | 19   | 8    | null | 10   | 56    |\\n| 12   | 2025-01-09| 14914| 29   | 63   | 546  | 23:55| 80   | null | 6    | 121  | Fri  | 1515 | 12   | 32   | 5    | 2644 | 13    |\\n| 58   | 48   | 2025-01-08| 8799 | 85   | 58   | 600  | 23:59| 96   | 2    | 7    | 158  | Sat  | 1908 | 14   | 21   | 1    | 1114  |\\n| 989  | 60   | 56   | 2025-01-07| 7206 | 18   | 36   | 492  | 22:46| null | 7    | 3    | 518  | Sun  | 1675 | 9    | 17   | 3     |\\n| 3    | 2144 | 14   | 39   | 2025-01-06| 10024| 61   | 21   | 442  | 23:58| 71   | 4    | 2    | null | null | 671  | 11   | 45    |\\n| 26   | null | 1960 | 48   | 21   | 2025-01-05| 398  | 31   | null | 580  | 22:58| 52   | 1    | 7    | 510  | Tue  | 1033 | 7     |\\n| 10   | 42   | 1    | 633  | 32   | 50   | 2025-01-04| 17948| 40   | 76   | 579  | 21:23| 22   | 8    | 8    | 959  | Wed  | 1804  |\\n| 888  | 10   | 55   | 1    | 2349 | 49   | 41   | 2025-01-03| null | 76   | 96   | 497  | 23:45| 12   | 9    | 7    | 553  | Thu   |\\n| Fri  | 1517 | 15   | 19   | 7    | 1942 | 56   | 59   | 2025-01-02| 4866 | 43   | 82   | 450  | 23:13| null | 2    | 6    | 291   |\\n| null | Sat  | 453  | null | 11   | null | 2745 | 23   | null | 2025-01-01| null | 13   | 52   | 596  | 23:35| 89   | 9    | null  |\\n| 10   | 912  | Sun  | null | 7    | 60   | 5    | 2678 | 17   | 15   | 2024-12-31| 4468 | 48   | 57   | 523  | 23:42| null | null  |\\n| 4    | 7    | 112  | null | null | 6    | 32   | 6    | 1987 | 54   | 13   | 2024-12-30| 9816 | null | 82   | 335  | 23:16| 70    |\\n| 10   | null | 2    | null | Tue  | 556  | 5    | 50   | 9    | null | 57   | 43   | 2024-12-29| 12057| null | 51   | null | 21:48 |.\\n\\nHowever, the code to convert the table into markdown format was buggy and, except for the header row that was formatted correctly, the values in the i-th row were rotated to the right i times (e.g., for the first row of values, the value at column j was moved to column j+1 and the value in the last column was moved to the first column). Compute the absolute difference between the sum of column sleep_minutes and column study_minutes, only looking at the days where the bed time was between 22:00 and 22:59 (inclusive) and the number of calls was greater than 2. Round the result to 2 decimal places. If it cannot be determined whether the condition is satisfied or not (e.g., due to null values that cannot be inferred from the rest of the table), assume the condition is not satisfied. If a value is null, ignore it from the computations. If any of the sums cannot be computed due to there being no values, the final answer must be \\\"0\\\". \"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}