{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3497607-fbff-5725-93a6-6b12e93c47e0","task_key":"default--test--5082","task_revision_id":"1","upstream_id":"5082","short_description":"The 'new racism' refers to:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['a subtler form of prejudice, masked by nationalist pride', 'a trend of neo-nazism masked as patriotism', 'a form of racism that only exists within the realm of social media', 'an anti-fascist movement which challenges nationalist politics', 'an increased awareness and sensitivity towards racism in contemporary society', 'a post-modern deconstruction of racist ideas to reveal their lack of depth', 'a radical form of racism that encourages open discrimination', 'a form of racism practiced by newly established countries', 'racist practices found in newly emerging areas of social life, such as cyberspace', 'the rise of racism in new political parties']\",\"question\":\"The 'new racism' refers to:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}