{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ef218de-e49f-5d11-9d91-7e59826528cf","task_key":"default--test--2215","task_revision_id":"1","upstream_id":"2215","short_description":"Carol and Jim are experiencing marital difficulties. They have two school-aged…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the girl is more likely than the boy to manifest an immediate behavior problem', 'the boy is less likely than the girl to manifest an immediate behavior problem', 'neither child is more likely to manifest an immediate behavior problem than are children of stable marriages', 'neither child is expected to manifest any behavior problem', 'the boy is more likely than the girl to manifest an immediate behavior problem', 'both children are less likely to manifest an immediate behavior problem than are children of stable marriages', 'neither child is less likely to manifest an immediate behavior problem than are children of stable marriages', 'the girl is less likely than the boy to manifest an immediate behavior problem', 'both children ore equally likely to manifest an immediate behavior problem', 'both children are more likely to manifest a delayed behavior problem']\",\"question\":\"Carol and Jim are experiencing marital difficulties. They have two school-aged children, a boy and a gid. According to meta-analysis of research on the relationship between marital discord and child behavior problems, it would be expected that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}