{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9672a65-fb9b-5928-98f6-2a065b92d51d","task_key":"default--test--11411","task_revision_id":"1","upstream_id":"11411","short_description":"A worm gear has a triple-threaded worm with a 2.5 in. lead and a gear of 3.5 in.…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['6.3 in.', '7.1 in.', '5.7 in.', '5.5 in.', '6.5 in.', '7.3 in.', '6.9 in.', '7.5 in.', '6.1 in.', '5.9 in.']\",\"question\":\"A worm gear has a triple-threaded worm with a 2.5 in. lead and a gear of 3.5 in. normaldiametralpitch. The number of teeth on the gear is 32. Determine the distance between the shaft centers and the pitch diameters of the worm and the gear.\",\"src\":\"stemez-MachineDesign\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}