{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e12fe24-4e59-5f57-9025-7d31c0a00313","task_key":"default--test--8148","task_revision_id":"1","upstream_id":"8148","short_description":"A field is in the shape of a trapezoid. The shorter base of the trapezoid is 130…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['24,375 yards^2', '34,125 yards^2', '43,875 yards^2', '48,750 yards^2', '9,750 yards^2', '29,250 yards^2', '58,500 yards^2', '19,500 yards^2', '14,625 yards^2', '39,000 yards^2']\",\"question\":\"A field is in the shape of a trapezoid. The shorter base of the trapezoid is 130 yards long and the longer base is 390 yards long. The distance between the two bases is 75 yards. What is the area of the field?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}