{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fa195576-00bd-5db9-b01f-d8e6ccdb8a2c","task_key":"default--test--10678","task_revision_id":"1","upstream_id":"10678","short_description":"Write the necessary statement function(s) to compute the area of a triangle,…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['S(A,B,C) = SQRT((A + B + C) / 2), AREA(A,B,C,X) = X*(X-A)*(X-B)*(X-C)', 'S(A,B,C) = (A + B + C) / 2, AREA(A,B,C,X) = SQRT(X*(X+A)*(X+B)*(X+C))', 'S(A,B,C) = (A + B + C), AREA(A,B,C,X) = SQRT((X-A)*(X-B)*(X-C))', 'S(A,B,C) = (A + B - C) / 2, AREA(A,B,C,X) = SQRT(X*(X-A)*(X-B)*(X-C))', 'S(A,B,C) = (A^2 + B^2 + C^2) / 2, AREA(A,B,C,X) = SQRT(X*(X-A)*(X-B)*(X-C))', 'S(A,B,C) = 0.5 * (A + B + C), AREA(A,B,C,X) = SQRT(X*(X-A)*(X-B)*(X-C))', 'AREA(A,B,C) = A*B*C', 'S(A,B,C) = (A + B + C) / 3, AREA(A,B,C,X) = SQRT(X*(X-A)*(X-B)*(X-C))', 'S(A,B,C) = (A + B + C) / 2, AREA(A,B,C,X) = X / ((X-A)*(X-B)*(X-C))', 'S(A,B,C) = A + B + C, AREA(A,B,C,X) = X*(X-A)*(X-B)*(X-C)']\",\"question\":\"Write the necessary statement function(s) to compute the area of a triangle, given 3 sides. Use Hero's formula.\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}