{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a29d1753-28a2-55ff-a27e-8f51fcaca186","task_key":"default--test--3073","task_revision_id":"1","upstream_id":"3073","short_description":"Pilocarpineis a drug that stimulates the nerve endings of parasympathetic…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['(a) Facilitates digestion, (b) Reduces pupil size, (c) Slows and weakens the heart beat', '(a) No effect on digestion, (b) Dilates pupil, (c) Speeds up heart beat', '(a) Slows digestion, (b) No effect on pupil, (c) Slows and weakens the heart beat', '(a) Facilitates digestion, (b) Increases pupil size, (c) Stabilizes heart beat', '(a) Inhibits digestion, (b) No effect on pupil, (c) Stabilizes heart beat', '(a) Slows digestion, (b) Increases pupil size, (c) Increases heart beat', '(a) No effect on digestion, (b) Reduces pupil size, (c) Increases heart beat', '(a) Inhibits digestion, (b) Increases pupil size, (c) Stabilizes heart beat', '(a) Facilitates digestion, (b) No effect on pupil, (c) Speeds up heart beat', '(a) Inhibits digestion, (b) Dilates pupil, (c) Slows and weakens the heart beat']\",\"question\":\"Pilocarpineis a drug that stimulates the nerve endings of parasympathetic nerves. What effects would you expect this drug to have on (a) the digestive tract, (b) the iris of the eye, and (c) the heart rate?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}