{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6e3ff67-d3be-5951-846c-13849f4a21a6","task_key":"default--test--7007","task_revision_id":"1","upstream_id":"7007","short_description":"How are craft unions different from industrial unions?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Industrial unions require a higher level of education for membership, while craft unions do not have educational requirements.', 'Craft unions join together workers with one particular skill, while industrial unions include workers in a particular plant or industry.', 'Craft unions are for skilled workers while industrial unions are for unskilled workers', 'Craft unions include workers from multiple industries, while industrial unions are specific to one craft or trade.', 'Craft unions are found only in the manufacturing sector, while industrial unions can be found in all sectors.', 'Industrial unions negotiate for individual workers, while craft unions negotiate for all workers within an industry.', 'Craft unions are smaller than industrial unions', 'Industrial unions are more powerful than craft unions', 'Craft unions focus on broad political agendas, while industrial unions focus solely on workplace conditions and pay.', 'Industrial unions represent freelance workers, while craft unions represent salaried employees.']\",\"question\":\"How are craft unions different from industrial unions?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}