{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3db387ea-069f-50b8-a712-3caed6a099bc","task_key":"default--test--11150","task_revision_id":"1","upstream_id":"11150","short_description":"By “animal motion,” Hobbes means:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['instinctive behavior, such as nursing young.', 'irrational behavior.', 'involuntary operations such as heartbeat and breathing.', 'the physical actions and reactions of animals.', 'human behavior that is driven by basic needs.', 'all forms of non-human animal behavior.', 'the process by which animals move from one place to another.', 'behavior that is driven by fear or survival instincts.', 'all voluntary behavior.', 'the movement and behavior of animals in the wild.']\",\"question\":\"By “animal motion,” Hobbes means:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}