{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3430ec80-0081-583c-b9ab-36eebeb3c4df","task_key":"default--test--2544","task_revision_id":"1","upstream_id":"2544","short_description":"If twosubculturalgroups have different mean IQ's (e.g., Blacks and whites) can…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Yes', 'Yes, but only within the same socioeconomic status', 'Only if the differences are consistent across all environments', 'Not always', 'No', 'Maybe, if other factors like education and access to resources are equal', 'Maybe', 'Only if twin studies support the heredity hypothesis']\",\"question\":\"If twosubculturalgroups have different mean IQ's (e.g., Blacks and whites) can it be concluded from that evidence that heredity is the most important and influential determinant of intelligence ?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}