{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e632843-373a-5853-9b67-b2722cbe2c44","task_key":"default--test--689","task_revision_id":"1","upstream_id":"689","short_description":"Which of the following is typically not a result of recognizing the importance…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Variations in product offerings to suit the wants of a particular group.', 'Use of an undifferentiated one-size-fits-all marketing strategy.', 'Promoting only mainstream products without considering ethnic preferences.', 'Adaptation of the advertising, particularly the use of foreign languages, to the target ethnic market.', 'Not considering the influence of ethnic backgrounds on purchasing decisions.', 'Assuming all ethnic groups have the same consumer behavior.', 'Ignoring the ethnic diversity in the target market.', 'Study of ethnic buying habits to isolate market segments.', 'Neglecting to conduct market research on ethnic groups.', 'Disregard of cultural sensitivities in advertising campaigns.']\",\"question\":\"Which of the following is typically not a result of recognizing the importance of ethnic groups by marketers?\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}