{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48155b19-d70e-5b6f-91a8-7486b9a12d80","task_key":"default--test--10579","task_revision_id":"1","upstream_id":"10579","short_description":"A PL/I program uses the EDIT form of input. The declare statementof the program…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['A -123.45, B 123.45, C 912.34, D 023.45, E 345.91, F 234.52, G BG2, H ACAR', 'A 123.45, B 123.45, C 912.34, D 456.00, E 123.45, F 018.34, G B2G, H CARS', 'A -123.45, B 123.45, C 023.40, D 123.45, E 011.83, F 900.90, G GB2, H ARSC', 'A 123.45, B 123.45, C 123.40, D -123.45, E 018.34, F 001.80, G 2GB, H SCAR', 'A -123.45, B 123.45, C 023.40, D 123.45, E 011.83, F 000.90, G B2G, H CARS', 'A -123.45, B 123.45, C 023.40, D 123.45, E 011.83, F 000.90, G 2BG, H CRSA', 'A 123.45, B -123.45, C 123.40, D 023.45, E 011.83, F 000.90, G GB2, H RASC', 'A -123.45, B 123.45, C 023.40, D 123.45, E 011.83, F 900.90, G B2G, H CARS', 'A -123.45, B 123.45, C 123.40, D 023.45, E 011.83, F 000.90, G B2G, H CARS', 'A 123.45, B -123.45, C 023.40, D 123.45, E 011.83, F 000.90, G B2G, H CARS']\",\"question\":\"A PL/I program uses the EDIT form of input. The declare statementof the program is as follows: DCL(A,B,C,D,E,F) FIXED(5,2) ; GCHAR(15)VAR, H CHAR(4) ; A data card used by the program has the following informationpunched on it (Note: The symbolbdenotes a blankspace). \\\\downarrowcol.8\\\\downarrowcol.34 AB-123 \\\\textbullet 45b123 \\\\textbullet 459123 \\\\textbullet 456bb12345b18 \\\\textbullet b 345912345B2GbCARS CARS CARS Show what will be stored in the memory locations correspond-ingto A, B, C, D, E, F, G and H when the followingstatement is executed: GETEDIT(A, B, C, D, E, F, G, H) [COL(10), F(7, 2), F(7, 2), X(2), F(4, 1), COL(34), F(5, 2), X(1), F(6, 3), F(1, 1), X(5), A(3), X(1), A(4)];\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}