{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a952421-ed20-54f0-ab4a-311e8af31545","task_key":"default--test--5803","task_revision_id":"1","upstream_id":"5803","short_description":"The media-themes model of media influence claims that:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['audiences are only influenced by the media if they actively seek out the message', 'the media only influences audiences who are already predisposed to agree with the message', 'audiences actively reject media messages that conflict with their pre-existing beliefs', 'audiences selectively interpret what they want to hear', 'media messages are universally interpreted in the same way by all audience members', 'media influence is only significant during major news events', 'content analysis is the best way to identify the themes covered by the media', 'audiences passively absorb whatever messages they are given', 'the media has no influence on audiences, as individuals form their own opinions independently', 'social interaction reinforces the ideas and images that audiences select']\",\"question\":\"The media-themes model of media influence claims that:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}