{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d2bdd8e-0331-5570-83c7-52d3781eea22","task_key":"default--test--75","task_revision_id":"3","upstream_id":"75","short_description":"Which of the following are the three broad groups of organizational…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Organizational size, industry type, and geographical location.', 'Organizational size, industry type, and age of company.', 'Organizational size, industry type, and customer base.', 'Organizational size, industry type, and annual revenue.', 'Organizational size, industry type, and business strategy.', 'None of the above.', 'Organizational size, industry type, and gender.', 'Organizational size, industry type, and number of branches.', 'Psychographics, purchaser, and behavioural criteria.', 'Organizational size, industry type, and number of employees.']\",\"question\":\" Which of the following are the three broad groups of organizational characteristics segmentation criteria?\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}