{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f3f61e7-1ba2-596a-bda1-0b961807015c","task_key":"default--test--9078","task_revision_id":"1","upstream_id":"9078","short_description":"A mass of 100 kilograms is pushed along the floor at a constant rate of 2 m/sec.…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['490 watts, 0.66 hp', '500 watts, 0.68 hp', '510 watts, 0.68 hp', '530 watts, 0.71 hp', '520 watts, 0.70 hp', '470 watts, 0.63 hp', '450 watts, 0.60 hp', '460 watts, 0.62 hp', '500 watts, 0.67 hp', '480 watts, 0.64 hp']\",\"question\":\"A mass of 100 kilograms is pushed along the floor at a constant rate of 2 m/sec. If the coefficient of sliding friction is 0.25, at what rate is work being done in watts, in horsepower?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}