{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a18d611d-789c-581e-8ea9-25d141792575","task_key":"all--test--a18d611d-789c-581e-8ea9-25d141792575","task_revision_id":"1","upstream_id":"","short_description":"Which of the following best describes the fallacy of poisoning the well?","config":"all","split":"test","body":"{\"choices\":\"['a way of suppressing evidence that eliminates a source of evidence from consideration by claiming the source is flawed, when there is no true relationship between the alleged flaw and the reliability of the source', 'arguing that a claim should be accepted based on evidence that is not presented, but asserted to be well known or obvious.', \\\"arguing something should be done based only on evidence that it's been done that way in the past, or should be believed only because it's been believed for a long time.\\\", 'reasoning that a claim is true based only on the evidence that it has been made so often.']\",\"question\":\"Which of the following best describes the fallacy of poisoning the well?\",\"subject\":\"logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}