{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa65ac31-3f4e-5821-a6f5-cc2bb8c8b727","task_key":"default--test--10914","task_revision_id":"1","upstream_id":"10914","short_description":"Carens's main conclusion is that","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['liberal egalitarian objections to anticosmopolitanism do not succeed.', 'Carens believes that liberal egalitarianism supports closed borders and strict immigration policies.', 'Carens argues that liberal egalitarianism is rooted in anti-immigration sentiments.', 'egalitarianism is founded on sentiments of envy and is therefore unjustified.', 'Carens concludes that liberal egalitarianism inherently supports anti-cosmopolitanism.', 'Carens concludes that egalitarianism is a justifiable sentiment, irrespective of immigration policies.', 'Carens asserts that liberal egalitarianism does not have a clear stance on immigration policies.', 'liberal egalitarianism should almost always press for more openness toward immigrants and refugees.', 'liberal egalitarianism, contrary to appearances, entails that only closed border immigration policies are acceptable.', 'Carens suggests that liberal egalitarianism always supports the concept of open borders.']\",\"question\":\" Carens's main conclusion is that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}