{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d66a8b8b-a404-5cda-953d-9de7dd9200f8","task_key":"default--test--11271","task_revision_id":"1","upstream_id":"11271","short_description":"Papadaki thinks that in cases where partners in a relationship mutually deny…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the context of the relationship is irrelevant in determining sexual objectification.', 'sexual objectification occurs.', 'the context of the relationship will determine whether or not sexual objectification occurred.', 'sexual objectification is only valid in a non-mutual context.', 'only physical abuse qualifies as sexual objectification.', 'both partners are incapable of sexually objectifying each other.', 'there is no straightforward answer to the question of whether or not sexual objectification occurs.', 'sexual objectification does not occur.', \\\"the partner who denies the other's humanity is always the one who sexually objectifies.\\\", 'neither partner is capable of sexual objectification.']\",\"question\":\" Papadaki thinks that in cases where partners in a relationship mutually deny each other's humanity that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}