{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"860505e7-5f07-515e-98fe-71d03a015915","task_key":"default--test--10872","task_revision_id":"1","upstream_id":"10872","short_description":"Singer presents his three-premise argument in addition to appealing to the case…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['he argues that moral claims can only be justified through logical reasoning and real-life examples.', 'he believes that abstract arguments are more likely to convince one to act.', 'he believes that using both abstract arguments and emotional appeals provides a balanced perspective.', \\\"he wants to appeal to the reader's reason and not just pull at the reader's heartstrings.\\\", 'he wants to show that moral obligations can arise from both rational argument and emotional response.', \\\"he believes arguments are a more efficient way to appeal to one's emotions.\\\", 'he believes the case of the drowning child will make his argument more relatable.', 'he argues that our emotional reactions have no bearing on which moral claims are true.', \\\"he aims to appeal to both the logical and emotional aspects of the reader's mind.\\\", 'he thinks that a combination of argument and emotion is more effective than either alone.']\",\"question\":\" Singer presents his three-premise argument in addition to appealing to the case of the drowning child because\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}