{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7871a3d6-6f2d-557e-bb74-143e5c673305","task_key":"default--test--12025","task_revision_id":"1","upstream_id":"12025","short_description":"A cast-iron flanged coupling is used to join two steel shafts 2(1/2) in., in…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['0.1280 in.', '0.1260 in.', '0.1220 in.', '0.1255 in.', '0.1215 in.', '0.1236 in.', '0.1250 in.', '0.1240 in.', '0.1200 in.', '0.1230 in.']\",\"question\":\"A cast-iron flanged coupling is used to join two steel shafts 2(1/2) in., in diameter, delivering a maximum torque of 60 ft.-lb. The two halves of the couplings are joined together getherby 4 bolts, placed equidistant on a bolt circle of 3 in. diameter. If the design stress is 10,000 psi in shear for the bolt material, evaluate the diameter of the bolts.\",\"src\":\"stemez-MachineDesign\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}