{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6dc9c75-3b06-5476-bd2b-6e12098611fc","task_key":"default--test--3450","task_revision_id":"1","upstream_id":"3450","short_description":"When an adult mouse or bird is castrated,itssexual behaviorvirtually disappears.…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Castration has no significant impact on sexual behavior in any animals, regardless of their level of development', 'Castration leads to an increase in sexual behavior in higher primates', 'Castration in higher primates reduces aggression but has no effect on sexual behavior', 'In higher primates such as man, castration does not affect the ability to perform the sexual act, provided the male is sexually mature and experienced. This is because the behavior has been learned through experience.', 'Castration in higher primates leads to a temporary disruption of sexual behavior, which is quickly restored', 'Castration stops all sexual behavior in higher primates', 'In higher primates, castration results in a change in the preference of sexual partners', 'Castration alters the learning ability of animals', 'Castration enhances the learning ability of animals regarding sexual behavior', 'Castration has the same effect in all animals']\",\"question\":\"When an adult mouse or bird is castrated,itssexual behaviorvirtually disappears. Compare the effects of castrationof highly developed primates with the less developedanimals.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}