{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"886bda8a-80d3-5bb6-bdfb-9c5aa5fdfb02","task_key":"default--test--5515","task_revision_id":"1","upstream_id":"5515","short_description":"Mass-society theory suggests that:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the subordinate classes are dominated by the ideology of the ruling class', 'the media serves as a platform for the masses to voice their opinions', \\\"the media manipulate 'the masses' as vulnerable, passive consumers\\\", 'audiences make selective interpretations of media messages', 'the content of the media is determined by market forces', \\\"the media's sole purpose is to disseminate information impartially\\\", 'the masses manipulate the media through their consumption patterns', 'the dominant classes are influenced by the ideologies of the subordinate class', 'the media is independent of societal influences', 'the content of the media is solely determined by government censorship']\",\"question\":\"Mass-society theory suggests that:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}