{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86924625-e267-5772-98f0-13ca361ad9fc","task_key":"all--test--86924625-e267-5772-98f0-13ca361ad9fc","task_revision_id":"1","upstream_id":"","short_description":"According to Corvino, the \"unnaturalness\" charge can carry moral weight only if…","config":"all","split":"test","body":"{\"choices\":\"['(i) explain the harms of anal sex and (ii) establish that all homosexuals will inevitably suffer those harms.', '(i) specify exactly what they mean by \\\"unnaturalness\\\" and (ii) explain why unnaturalness (so defined) matters morally.', \\\"(i) establish that one's sexual orientation is a matter of choice and (ii) that people who choose homosexuality have an intrinsic desire for self-harm.\\\", '(i) explain the value of naturalness and (ii) why nothing \\\"unnatural\\\" can be of value.']\",\"question\":\" According to Corvino, the \\\"unnaturalness\\\" charge can carry moral weight only if those who level it can do two things:\",\"subject\":\"moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}