{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"52508bfd-1430-52d1-96da-d8523917fa7f","task_key":"default--test--10836","task_revision_id":"1","upstream_id":"10836","short_description":"The _ad nauseam_ fallacy is","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"reasoning that a conclusion is false because it's been claimed so rarely\\\", \\\"reasoning that a claim is true because it hasn't been disproven\\\", 'arguing that an act should be carried out because it will not have any significant consequences, when there is no evidence of the effects', \\\"arguing that a conclusion is true because it's been disproven so often\\\", \\\"arguing that something should be done because that's the way it's been done in the past.\\\", \\\"reasoning that a conclusion is true because it's been claimed so often\\\", 'reasoning that a claim is false because so many people believe it is false', 'arguing that an act should be avoided because it will have significant consequences, when there is no evidence of the effects', 'reasoning that a claim is true because so many people believe it is true', \\\"arguing that something should not be done because it's never been done in the past\\\"]\",\"question\":\"The _ad nauseam_ fallacy is\",\"src\":\"ori_mmlu-logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}