{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88013374-bfe1-56d1-9ffd-525c643fdb56","task_key":"default--test--2478","task_revision_id":"1","upstream_id":"2478","short_description":"If a symptom does not succeed in relieving anxiety does the neurotic discard it?","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the neurotic discards them', 'The neurotic accepts them but seeks additional solutions', 'the neurotic ignores them', \\\"The neurotic is unaware of the symptom's presence\\\", 'the neurotic does not discard them', 'The neurotic suppresses them until they are effective']\",\"question\":\"If a symptom does not succeed in relieving anxiety does the neurotic discard it?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}