{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fef989f5-1384-53c1-9b1b-c31f8f2f3ebc","task_key":"default--test--5805","task_revision_id":"1","upstream_id":"5805","short_description":"Sociobiologists believe that there are more males than females with paraphilias…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['paraphilias guarantee survival of the species', 'of hormonal differences between the genders', 'females have evolved to suppress paraphilias', 'males are more prone to genetic mutations that lead to paraphilias', 'males are more likely to report paraphilias due to societal pressures', 'males are more likely to be influenced by environmental factors', 'of left brain/right brain differences between the genders', 'male brains are better developed than are female brains', 'evolutionary selection has favored the development of males who respond to novel sexual stimuli', 'sociobiologists do not believe that there are more males than females with paraphilias']\",\"question\":\"Sociobiologists believe that there are more males than females with paraphilias because:\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}