{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c25778f-6378-57e3-ad0b-01a59b8fef19","task_key":"default--test--10996","task_revision_id":"1","upstream_id":"10996","short_description":"Singer argues that developing nations are harming poor nations by","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['exploiting their natural resources without adequate compensation.', 'indoctrinating them with Western values.', 'forcing them to adopt Western educational systems.', 'imposing their political systems on them.', 'promoting consumerism and materialistic values.', 'causing global warming.', 'ignoring important aspects of their culture.', 'refusing to help with debt relief.', 'imposing trade restrictions and tariffs.', 'handing out too much foreign aid, which increases need.']\",\"question\":\" Singer argues that developing nations are harming poor nations by\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}