{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"646895b1-3345-525d-aad8-cf8cd4b3027f","task_key":"default--test--12172","task_revision_id":"1","upstream_id":"12172","short_description":"A power of 30 hp at 150 rpm is available on a distribution line shaft of 2.5 in…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['0.21 degree', '0.31 degree', '0.36 degree', '0.71 degree', '0.45 degree', '0.26 degree', '0.55 degree', '0.51 degree', '0.61 degree', '0.41 degree']\",\"question\":\"A power of 30 hp at 150 rpm is available on a distribution line shaft of 2.5 in diameter. Two pulleys mounted on the shaft, for power distribution, 18 in. apart tend to cause torsionaldeflection. Calculate the angle of twist in the shaft between the pulleys. Shear modulus of elasticity = 11 × 10^6 psi.\",\"src\":\"stemez-MachineDesign\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}