{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4da59348-bc33-50ee-bfe4-18b260f02c01","task_key":"default--test--10992","task_revision_id":"1","upstream_id":"10992","short_description":"On West's account, women who engage in undesired sex because of actual or felt…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['they are dominated by their partners insofar as they are not free to do as they please.', 'they are actually coerced into having sex.', 'they are financially dependent on their partners.', 'they fear the consequences of refusing their partner.', 'they have not established boundaries in their relationship.', 'none of the above.', 'their partner uses manipulation tactics to coerce them into sex.', \\\"they feel obliged to fulfill their partner's sexual desires.\\\", 'they feel a sense of obligation due to societal pressures.', 'they have neglected to take whatever steps would lead to their self-sustenance.']\",\"question\":\" On West's account, women who engage in undesired sex because of actual or felt dependence on their partner may experience injury to their sense of autonomy because\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}