{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c589bbe-dd81-5dbc-ba20-1378df96df6c","task_key":"default--test--12175","task_revision_id":"1","upstream_id":"12175","short_description":"A load of 7.5 tons is to be lifted from a depth of 760 ft. using a wire rope. It…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['Both the rope and the drum sizes are not acceptable', 'Both the rope and the drum sizes are acceptable, but the rope material needs to be upgraded', 'The size of the drum is not acceptable', 'Both the rope and the drum sizes are acceptable, but the drum material needs to be upgraded', 'The size of the rope is acceptable, but the drum should be larger than 96 inches in diameter', 'The size of the rope is not acceptable', 'The size of the rope is acceptable, but the drum should be smaller than 96 inches in diameter', 'The size of the drum is acceptable, but the rope should be thinner than 1(1/2) inches in diameter', 'The size of the rope and the drum are acceptable', 'The size of the drum is acceptable, but the rope should be thicker than 1(1/2) inches in diameter']\",\"question\":\"A load of 7.5 tons is to be lifted from a depth of 760 ft. using a wire rope. It is suggested to use a widely applied 6 × 19 IPS rope of diameter 1(1/2) in. and drum of diameter 96 in. Check if these sizes are suitable.\",\"src\":\"stemez-MachineDesign\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}