{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"53a0bbe7-bea6-5b85-9ada-a2351ff3e4a2","task_key":"default--test--6087","task_revision_id":"1","upstream_id":"6087","short_description":"Which of these features suggest that a pulsation in the neck is arterial?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Positive for hepatojugular reflux', 'Pulsation can be visually seen through the skin', 'The pulsation is erratic and inconsistent', 'No variation with posture', 'The pulsation is felt on both sides of the neck', 'Pulsation increases with physical activity', 'No variation with breathing', 'A waggling earlobe', 'Pressure in the root of the neck reduces the impulse', 'Pressure in the mid-section reduces the impulse']\",\"question\":\"Which of these features suggest that a pulsation in the neck is arterial?\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}