{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba69c761-554f-55b5-8ee4-e3e3fdf03a1f","task_key":"default--test--5946","task_revision_id":"1","upstream_id":"5946","short_description":"Chodorow (1978) argued that gender socialization occurred through:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['both boys and girls being closely attached to their parents, but then girls breaking away', 'both boys and girls being closely attached to their fathers, but then boys breaking away', 'both boys and girls being closely attached to their fathers, but girls breaking away', 'girls being attached to their mothers but then breaking away', 'both boys and girls being closely attached to their mothers, but then boys breaking away', 'boys being closely attached to their mothers, with girls breaking away', 'boys being attached to their mothers but then breaking away', 'both boys and girls being closely attached to their siblings', 'boys being attached to their fathers and girls to their mothers', 'girls being attached to their fathers and boys to their mothers']\",\"question\":\"Chodorow (1978) argued that gender socialization occurred through:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}