{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df741572-76ce-5fd8-b5e6-6f81e1e1fe1d","task_key":"default--test--10812","task_revision_id":"1","upstream_id":"10812","short_description":"One example of bad constitutive moral luck would be:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['a person who donates to charity regularly.', 'all of the above.', 'a person who lived in Nazi Germany in 1940.', 'a person who drives drunk and hits a child.', 'none of the above.', 'a person who is naturally kind and generous.', 'a person who lived in the United States in the 1960s.', 'a person who drives carefully and avoids accidents.', 'a person who is by nature envious and ungenerous.', 'a person who lives in a peaceful, affluent country.']\",\"question\":\"One example of bad constitutive moral luck would be:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}