{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40fcf13c-dcf0-56b3-b70b-c3dec4256cbd","task_key":"default--test--2376","task_revision_id":"1","upstream_id":"2376","short_description":"Outline and explain some of the variables which increase or decreaseconformity.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Education level, career, and gender', 'Socio-economic status, personality type, and age', 'Religion, political belief, and family background', 'The time of day, weather conditions, and geographical location', 'Clothing choices, music preferences, and favorite foods', \\\"Majority opinion's unanimity, prior commitment to the initial judgement, group composition, and individual's self-esteem.\\\", 'The language spoken, hair color, and hobbies']\",\"question\":\"Outline and explain some of the variables which increase or decreaseconformity.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}