{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"37000e8b-0c23-5911-b32d-9e5232321cf9","task_key":"all--test--37000e8b-0c23-5911-b32d-9e5232321cf9","task_revision_id":"1","upstream_id":"","short_description":"Which of the following is one of de Marneffe's responses to the objection that…","config":"all","split":"test","body":"{\"choices\":\"['Drug prohibition would be done for the benefit of others, not the benefit of those who wish to produce/sell drugs, so it is not paternalistic in an objectionable sense.', 'Drug prohibition would be paternalistic, but its consequences would be good enough to outweigh considerations about paternalism.', 'Drug prohibition would be targeted to adults, and one can only be paternalistic toward children.', 'Paternalism is always justified when the paternalistic practices truly are in the best interest of those affected.']\",\"question\":\" Which of the following is one of de Marneffe's responses to the objection that drug prohibition would be paternalistic?\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}