{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ec23d62-ad75-5568-9bb9-aea0ffb651cd","task_key":"default--test--5154","task_revision_id":"1","upstream_id":"5154","short_description":"In their study of depression, Brown & Harris (1978) found that:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['men and women have fairly similar levels of diagnosed depression', 'depression is less likely in individuals with high levels of social support regardless of gender', \\\"women's susceptibility to stress depended on their levels of social support\\\", 'men and women have drastically different levels of diagnosed depression', \\\"men's susceptibility to stress depended on their levels of social support\\\", 'depression is more common in women due to hormonal differences', 'women are more prone to depression due to societal pressures', 'men are more vulnerable to depression because of conventional gender roles', 'self-reported depression is highest amongst women and the economically disadvantaged', 'self-reported depression is highest amongst men and the relatively affluent']\",\"question\":\"In their study of depression, Brown & Harris (1978) found that:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}