{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d40c9fb2-6232-5042-a2e0-a83c9370872c","task_key":"default--test--11233","task_revision_id":"1","upstream_id":"11233","short_description":"In the context of Gardiner's article, \"the fragmentation of agency\" refers to…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"an agent's inability to acknowledge his or her contribution to climate change\\\", 'the fragmentation of the effects of greenhouse gas emissions across generations', 'the vast number of individuals and agents who contribute to climate change', \\\"the idea that it's not rational for each individual to restrict his or her own pollution\\\"]\",\"question\":\" In the context of Gardiner's article, \\\"the fragmentation of agency\\\" refers to what?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}