{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90b9716e-cd28-5de7-bb30-fd5d8eb53fd7","task_key":"default--test--10928","task_revision_id":"1","upstream_id":"10928","short_description":"Which of the following describes he fallacy of appeal to pride?","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['arguing someone should accept a claim based only on the evidence that he or she has the favorable characteristics to understand the truth of the claim.', \\\"arguing that a claim should be accepted based only on the evidence that it's in line with traditional or cultural beliefs.\\\", \\\"arguing that a claim should be accepted based only on the evidence that it's supported by a famous or respected figure.\\\", 'arguing that a claim should be accepted based only on the evidence that rejecting it will lead to negative consequences.', 'arguing that an act or series of acts should be carried out because they will have no significant consequences, when there is no evidence for the lack of significant effects.', \\\"arguing something should be done based only on evidence that it's been done that way in the past, or should be believed only because it's been believed for a long time.\\\", \\\"arguing that a claim should be accepted based only on the evidence that it's widely accepted by the public.\\\", \\\"arguing that a claim should be accepted based only on the evidence that it's appealing or desirable.\\\", 'arguing a claim should be accepted based only on evidence that unidentified authorities accept it.', \\\"arguing that a claim should be accepted based only on the evidence that it's been scientifically proven, without providing the actual proof.\\\"]\",\"question\":\"Which of the following describes he fallacy of appeal to pride?\",\"src\":\"ori_mmlu-logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}