{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2bce46ae-9543-59b9-9896-473cd6c15d74","task_key":"default--test--2927","task_revision_id":"1","upstream_id":"2927","short_description":"Which of the following is the correct characteristic of arteries?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They carry blood towards the heart.', 'They always carry deoxygenated blood.', 'They carry blood away from the heart.', 'They contain valves that prevent backflow.', 'They always carry oxygenated blood.', 'They have a large lumen compared to veins.', 'They are responsible for the exchange of gases and nutrients.', 'They form a network of small vessels called capillaries.', 'They are thin-walled blood vessels.', 'They contain no muscle tissue.']\",\"question\":\"Which of the following is the correct characteristic of arteries?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}