{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1f2717a4-1c5c-50ac-b9c0-62b1708cb253","task_key":"default--test--5373","task_revision_id":"1","upstream_id":"5373","short_description":"In Symbolic Interactionist theory, Mead defined the 'generalized other' as:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['a collection of individual roles and expectations in society', 'the viewpoint of a specific other individual we interact with', 'an imaginary audience that is always judging our actions', 'the overall impression of ourselves that we try to give off to others', 'an image of how people in the wider society might perceive our behaviour', 'a significant figure in early childhood who teaches us the general values of society', 'the group of structural theories of society that he was reacting against', 'the inherent human tendency to seek social acceptance']\",\"question\":\"In Symbolic Interactionist theory, Mead defined the 'generalized other' as:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}