{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5fd8c030-93ca-5f0b-8005-9900f648e597","task_key":"all--test--5fd8c030-93ca-5f0b-8005-9900f648e597","task_revision_id":"1","upstream_id":"","short_description":"Which of the following describes the fallacy of appeal to common practice?","config":"all","split":"test","body":"{\"choices\":\"['eliminating a source of evidence from consideration by claiming the source is flawed, when there is no true relationship between the alleged flaw and the reliability of the source', 'reasoning that an act or series of acts should be carried out because they will have no significant consequences, when there is no evidence there will be no consequences', 'drawing a conclusion after purposely leaving out evidence or ignoring known evidence that might contradict a claim', 'arguing that something is the right thing to do based only on evidence that other people are already doing it.']\",\"question\":\"Which of the following describes the fallacy of appeal to common practice?\",\"subject\":\"logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}