{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54a83298-95f2-539a-a383-1994c58165d4","task_key":"default--test--11412","task_revision_id":"1","upstream_id":"11412","short_description":"Calculate the number of coils of a helical spring so that the deflection of the…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['10', '13', '14', '15', '12', '7.5', '8', '10.5', '9', '11.5']\",\"question\":\"Calculate the number of coils of a helical spring so that the deflection of the spring may not exceed 1.5 in. under a certain load. The mean diameter of the spring is 2.25 in. and the wire diameter is 0.15 in., with the material having an allowable shear stress of 50,000 psi and shear modulus of elasticity of 12 × 10^6.\",\"src\":\"stemez-MachineDesign\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}