{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48846f6d-c84b-58c2-8ab6-a16eb759a7ee","task_key":"default--test--5558","task_revision_id":"1","upstream_id":"5558","short_description":"Which of the following statements is correct regarding the Federal Unemployment…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The unemployment insurance system is administered by the states through their employment laws.', 'The Act is intended to provide financial assistance to unemployed workers who have been laid off due to seasonal changes in demand.', 'An employee who resigns regardless of cause is eligible for unemployment benefits.', 'An employee who is fired for cause is eligible for unemployment benefits.', 'The Act is intended to assist workers who are permanently out of work and need assistance in supporting themselves.', 'The Act mandates that employers provide health insurance to employees who are unemployed.', 'The federal unemployment system is funded solely by employee taxes.', 'The federal unemployment system is funded by both employer and employee taxes.', 'The Act also covers compensation for self-employed individuals.', 'The Act is enforced by the Federal Labor Relations Authority.']\",\"question\":\"Which of the following statements is correct regarding the Federal Unemployment Tax Act?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}