{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"82e3b9dc-5850-509e-8797-42eb093842c1","task_key":"default--test--11247","task_revision_id":"1","upstream_id":"11247","short_description":"Which of the following explanations best fits the fallacy of ignorance of…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"asserting an argumentative opponent has taken an easily defeated position, which the opponent has not really taken, defeating the position, and acting as if you've done significant damage to that person's overall argument.\\\", 'causing confusion during refutation because of a real or feigned lack of the ability to engage in refutation', 'characterizing an opposing argument in such a way to make it or its consequences appear to be ridiculous', 'arguing an action should be taken based only on the need to be loyal to someone or to a group']\",\"question\":\"Which of the following explanations best fits the fallacy of ignorance of refutation?\",\"src\":\"ori_mmlu-logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}