{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ac7485a-57bd-5a92-a501-a7d3b11a0b36","task_key":"default--test--10835","task_revision_id":"1","upstream_id":"10835","short_description":"The false analogy fallacy is","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['a specific type of false cause fallacy that argues that since two event happened at the same time one event must have caused the other', 'arguing that a statement is true because it has not been proven false', 'a specific type of false cause fallacy that argues that since two events happened in sequence, the first event must have caused the second.', 'arguing that a statement is false because it has not been proven true', 'arguing that since a statement is popular, it must be true', 'drawing a conclusion based on a comparison when the items being compared are not similar enough to support the conclusion.', 'arguing that since something is new it is necessarily good', 'drawing a conclusion based on a comparison when the items being compared are too similar to distinguish', 'a fallacy of relevance where the premise of an argument is irrelevant to the conclusion', 'arguing that an event that preceded another event necessarily is the cause of the second event']\",\"question\":\"The false analogy fallacy is\",\"src\":\"ori_mmlu-logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}