{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ea2e1878-dde7-5096-bbfd-d4c6dc02b045","task_key":"default--test--11132","task_revision_id":"1","upstream_id":"11132","short_description":"Since we who live in wealthy countries have directly caused harm, Pogge argues…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['libertarian duties towards the global poor', 'positive duties towards the global poor', 'duty-free towards the global poor', 'liberal duties towards the global poor', 'consequentialist duties towards the global poor', 'conservative duties towards the global poor', 'negative dutiestowards the global poor', 'utilitarian duties towards the global poor', 'human rights duties towards the global poor', 'egalitarian duties towards the global poor.']\",\"question\":\" Since we who live in wealthy countries have directly caused harm, Pogge argues that we have very stringent\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}