{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7f3bd1a-578a-5462-bbfa-b78b690736d8","task_key":"default--test--10958","task_revision_id":"1","upstream_id":"10958","short_description":"Foot's Rescue I and Rescue II cases are intended to","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['contrast originating a harmful sequence with allowing a harmful sequence to continue.', 'all of the above', 'demonstrate the difference between passive and active harm.', 'argue that good intentions can result in harmful sequences.', 'show that harmful sequences can be stopped by good intentions.', 'contrast intentional harm with accidental harm.', 'argue that all harmful sequences are the result of evil intentions.', 'highlight the importance of intentions in determining the morality of an action.', 'show that evil intentions always involve the origination of harmful sequences.', 'contrast evil intentions with good intentions.']\",\"question\":\" Foot's Rescue I and Rescue II cases are intended to\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}