{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe9535cf-a6a8-5170-a149-953a585d72eb","task_key":"default--test--1573","task_revision_id":"1","upstream_id":"1573","short_description":"Why does Parfit oppose equality?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['He claims that by giving priority to the needs of the poor, we can increase equality.', 'He rejects the idea of equality altogether.', 'He believes that equality is not a realistic goal.', 'He argues than an unequal society is inevitable.', 'He suggests that societal structures inherently support inequality.', 'He asserts that we are each responsible for our poverty.', 'He posits that striving for equality undermines individual freedoms.', 'He argues that inequality is necessary for competition and progress.', 'He contends that inequality is a natural human condition.']\",\"question\":\"Why does Parfit oppose equality?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}