{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee1979dd-9bdb-5c50-a200-a1d14a94f6fc","task_key":"default--test--2873","task_revision_id":"1","upstream_id":"2873","short_description":"A behavioral response called a fixed action pattern shown by animals","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['is a learned behavior that can be unlearned', 'occurs the second time an animal is exposed to the correct stimulus at the appropriate time in its life', \\\"only occurs during the animal's sleep cycle\\\", 'is a response that only occurs in the presence of other animals', \\\"is a response that is influenced by the animal's diet\\\", 'is a response that develops as the animal ages', \\\"is triggered by a number of sensory signals in the animal's environment\\\", 'is a motor response which once released may be terminated spontaneously', 'occurs in the absence of sensory feedback', 'is a reflex action triggered by external stimuli']\",\"question\":\"A behavioral response called a fixed action pattern shown by animals\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}