{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2a03c89-47c8-5258-b77e-ffdde1d60681","task_key":"default--test--2192","task_revision_id":"1","upstream_id":"2192","short_description":"Discuss the three attributes of color. Why do red and pink appearto be different…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Saturation, tone, and pigment', 'Intensity, shade, and tone', 'Chroma, contrast, and clarity', 'Brightness, shade, and chroma', 'Hue, brightness and saturation', 'Pigment, texture, and luminance', 'Tone, shade and tint', 'Red, green and blue', 'Hue, saturation, and value', 'Contrast, brightness and hue']\",\"question\":\"Discuss the three attributes of color. Why do red and pink appearto be different colors ?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}