{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70b39ec6-fd38-53d9-bca4-88e65f18d627","task_key":"default--test--2691","task_revision_id":"1","upstream_id":"2691","short_description":"You are a consultant asked to design an intervention program in a recently…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['for each group to reinforce their own positive traits without considering the other', 'where old and new residents are separated during most interactions', \\\"which encourage residents to ignore each other's presence\\\", 'which provide an open airing and all-out discussion about what each group believes to be the negative crits of the other', 'which make explicit the status hierarchy chat is implicit in the interactions between the wo groups', 'that allow residents to vent their frustrations without any moderation', 'where residents can engage in competitive activities to establish dominance', 'for each group of residents co disconfirm negative traits of the other group', 'that involve only the leaders of each resident group in discussions']\",\"question\":\"You are a consultant asked to design an intervention program in a recently integrated housing project where antagonism and hostility between old and new residents are extremely high. The goal is to improve relations through contact. You would be more likely to facilitate that goal if you could arrange interactions that provide opportunities\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}