{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d00e50e-5c84-5efb-a05d-18c9a07b6cc8","task_key":"default--test--12026","task_revision_id":"1","upstream_id":"12026","short_description":"An electric saw uses a no. 50 chain of pitch 5/8 in. that runs onsprockets; the…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['16.33 hp and 14.966 hp', '12.33 hp and 10.966 hp', '14.33 hp and 12.966 hp', '8.33 hp and 6.966 hp', '13.33 hp and 11.966 hp', '15.33 hp and 13.966 hp', '7.33 hp and 5.966 hp', '9.33 hp and 7.966 hp', '10.33 hp and 8.966 hp', '11.33 hp and 9.966 hp']\",\"question\":\"An electric saw uses a no. 50 chain of pitch 5/8 in. that runs onsprockets; the smaller having 18 teeth. For conditions of smoothoperation, calculate the horsepower transmitted if, (a)smallersprocket rotates at 1000 rpm. (b)smallersprocket rotates at 2000 rpm.\",\"src\":\"stemez-MachineDesign\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}