{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2b2ba3e-df92-5561-b8af-b260be4467b0","task_key":"default--test--2454","task_revision_id":"1","upstream_id":"2454","short_description":"A community bank consists of four branch offices with approximately 60 employees…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Development of, and participation in, local community group sessions focusing on weight loss and smoking cessation', 'Employee reimbursement for costs and fees associated with professional help for weight loss and smoking cessation', 'A competition among the four branches focusing on stepwise reductions in weight and smoking', 'Providing gym memberships and nicotine patches to all employees', 'Instituting a penalty system for employees who do not lose weight or quit smoking', 'Mandatory weight loss and smoking cessation classes during work hours', 'Distribution of health information and self-help materials related to weight loss and smoking cessation', 'Implementing a company-wide ban on smoking and unhealthy food in the office.']\",\"question\":\"A community bank consists of four branch offices with approximately 60 employees each. The general management would like to institute 2 health risk-reduction program by encouraging weight loss and smoking cessation among the employees. Which of the following programs would be most effective\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}