{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f46180bd-02b7-59b2-b096-6f61be031770","task_key":"default--test--10805","task_revision_id":"1","upstream_id":"10805","short_description":"Which of the following describes the straw person fallacy?","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['arguing someone should do something only because it will make him or her better liked', \\\"saying an opposing arguer has taken a position that you can easily defeat, when he or she has not taken that position, then going on to defeat that position and claim to have done significant damage to that arguer's overall argument\\\", \\\"attacking the character or background of an opposing arguer instead of responding to that person's arguments with good evidence and sound reasoni\\\", 'responding to an argument with an expression of derision instead of a counter argument']\",\"question\":\"Which of the following describes the straw person fallacy?\",\"src\":\"ori_mmlu-logical_fallacies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}