{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"afa0ec43-4c42-5971-bd1f-0b749f8bda71","task_key":"default--test--10783","task_revision_id":"1","upstream_id":"10783","short_description":"Ashford's article is meant to address a particular paralysis in the face of","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the daunting task of solving worldwide economic imbalance.', \\\"the impossibility of meeting everyone's basic needs.\\\", \\\"having to give up so much of our own wealth in light of Singer's arguments.\\\", 'having to choose between addressing immediate harm and addressing structural injustice.', 'reconciling conflict moral theories.', 'the difficult task of prioritizing multiple global issues.', 'the challenge of implementing sustainable solutions to poverty.', 'the sheer scope of global poverty.', 'the overwhelming responsibility of addressing world hunger.', 'the complexity of addressing environmental issues globally.']\",\"question\":\" Ashford's article is meant to address a particular paralysis in the face of\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}