{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5affee53-6fd7-537f-822f-9b672fec9148","task_key":"all--test--5affee53-6fd7-537f-822f-9b672fec9148","task_revision_id":"1","upstream_id":"","short_description":"Which of the following explanations best fits the fallacy of ignorance of…","config":"all","split":"test","body":"{\"choices\":\"[\\\"asserting an argumentative opponent has taken an easily defeated position, which the opponent has not really taken, defeating the position, and acting as if you've done significant damage to that person's overall argument.\\\", 'causing confusion during refutation because of a real or feigned lack of the ability to engage in refutation', 'characterizing an opposing argument in such a way to make it or its consequences appear to be ridiculous', 'arguing an action should be taken based only on the need to be loyal to someone or to a group']\",\"question\":\"Which of the following explanations best fits the fallacy of ignorance of refutation?\",\"subject\":\"logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}