{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e941d328-58e2-5876-99b7-6e767635b901","task_key":"default--test--2199","task_revision_id":"1","upstream_id":"2199","short_description":"With regard to the effects of crowding, which of the following is true?","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Men and women react aggressively to crowded conditions to about the same degree', 'Men and women are equally likely to react negatively to crowded conditions.', 'Men and women are equally likely to act aggressively in crowded conditions.', 'Men are less likely than women to be stressed by crowded conditions.', 'Women are less likely than men to be stressed by crowded conditions.', 'Women are more likely than men to act aggressively in crowded conditions.', 'Women and men are equally unaffected by crowded conditions.', 'Men react to crowded conditions with stress more than women.', 'Women react aggressively to crowded conditions to a greater degree than men.']\",\"question\":\"With regard to the effects of crowding, which of the following is true?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}