{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d13e65d2-f397-5f69-8ae3-50e62a5d73d8","task_key":"default--test--11195","task_revision_id":"1","upstream_id":"11195","short_description":"Ashford argues that worries about causing dependency on aid agencies among those…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['suggest that aid agencies are the only solution to global poverty.', \\\"abandon the world's poor to their unjust predicament.\\\", 'argue that aid agencies are inherently unjust.', \\\"increase the amount of aid given to the world's poor.\\\", 'ignore the issue of global economic justice.', \\\"force the world's poor to be self-sufficient.\\\", 'stop all forms of international aid.', 'accept the view that we have no negative duties to the poor.', 'adopt a social contract approach to the problem of global economic justice.', 'legally abolish aid agencies.']\",\"question\":\" Ashford argues that worries about causing dependency on aid agencies among those in extreme poverty does not mean that we should\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}