{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"194b589f-b155-5b76-99b8-40d3f4ce4556","task_key":"default--test--10909","task_revision_id":"1","upstream_id":"10909","short_description":"Which of the following is one of de Marneffe's responses to the objection that…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['Paternalism is always justified when the paternalistic practices truly are in the best interest of those affected.', 'Drug prohibition would be paternalistic, but its consequences would be good enough to outweigh considerations about paternalism.', 'Paternalism is a positive trait when it helps guide individuals away from harmful behaviors like drug use.', 'Drug prohibition would be a form of societal control, not paternalism.', 'Drug prohibition would be targeted to adults, and one can only be paternalistic toward children.', 'Drug prohibition would be paternalistic, but it is a necessary evil to maintain order and protect public health.', \\\"The concept of paternalism doesn't apply to the issue of drug prohibition.\\\", \\\"Drug prohibition is not paternalistic as it does not impose a 'parent-like' authority over individuals.\\\", 'Drug prohibition would not be paternalistic because it would protect society from the harmful effects of drugs.', 'Drug prohibition would be done for the benefit of others, not the benefit of those who wish to produce/sell drugs, so it is not paternalistic in an objectionable sense.']\",\"question\":\" Which of the following is one of de Marneffe's responses to the objection that drug prohibition would be paternalistic?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}