{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"606aff02-c22d-511e-81cd-0640fbf701bf","task_key":"default--test--11171","task_revision_id":"1","upstream_id":"11171","short_description":"Which of the following describes the horse laugh fallacy?","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['arguing actions should be taken to avoid negative results, when those results are exaggerated, unlikely, or irrelevant', \\\"claiming that an argument is invalid based on the speaker's hypocrisy or inconsistency\\\", \\\"characterizing an opponent's position in such way to make it or its consequences appear to be ridiculous\\\", 'responding to an argument with an expression of derision instead of a counter argument', 'arguing that since something has not been proven false, it must be true', \\\"attacking an opponent's character or personal traits instead of their argument\\\", 'using personal feelings or prejudices to sway the argument, instead of logic', \\\"using a misrepresentation or exaggeration of an opponent's argument to make it easier to attack\\\", 'attempts to gain agreement based solely the feelings aroused by the message', 'introducing a point that is irrelevant to the argument as a diversionary tactic']\",\"question\":\"Which of the following describes the horse laugh fallacy?\",\"src\":\"ori_mmlu-logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}