{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"031e4241-b4fa-50d1-a05a-1162f26360ae","task_key":"default--test--5950","task_revision_id":"1","upstream_id":"5950","short_description":"What seems to be the case when the folk saying \"opposites attract\" (i.e.,…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['It seems to be true for heterosexuals but not for homosexuals.', 'It seems to be true for men but not for women.', 'It seems to be true for homosexuals but not for heterosexuals.', 'It seems to be true for women but not for men', 'It seems to be true for people with certain personality traits.', 'It seems to be true in some cultures but not in others.', 'It does not seem to be true.', 'It seems to be true only in certain situations.', 'It seems to be true for older adults but not for younger adults.', 'It seems to be true for younger adults but not for older adults.']\",\"question\":\"What seems to be the case when the folk saying \\\"opposites attract\\\" (i.e., dissimilar attitudes tend to cause liking) when it is considered in light of current research?\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}