{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b34ece5-234e-5cc5-afe7-2e43ebd73956","task_key":"default--test--5217","task_revision_id":"1","upstream_id":"5217","short_description":"Joe wants desperately to get close to Helen. Helen appears to be put off by…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['a detached lover', 'an avoidant lover', 'a disorganized lover', 'a fearful-avoidant lover', 'a secure-avoidant lover', 'a passionate lover', 'a dismissive-avoidant lover', 'a secure lover', 'an obsessive lover', 'an anxious-ambivalent lover']\",\"question\":\"Joe wants desperately to get close to Helen. Helen appears to be put off by Joe's desperate advances. This is typical of Joe's relationships with women. According to attachment theory Joe would best be described as:\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}