{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a511f41d-d3b6-5ddc-ba9b-295d365e4947","task_key":"default--test--6418","task_revision_id":"1","upstream_id":"6418","short_description":"You notice that your patient's submandibular lymph nodes are enlarged. You would…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['hard palate.', 'hard palate, upper lip, and upper central incisor.', 'upper lip and upper central incisor.', 'hard palate, upper lip, and lower first molar.', 'hard palate and upper lip.', 'hard palate, upper lip and upper central incisor.', 'hard palate, upper central incisor and lower first molar.', 'hard palate, upper lip, upper central incisor and lower first molar.', 'upper lip and lower first molar.', 'upper lip, upper central incisor and lower first molar.']\",\"question\":\"You notice that your patient's submandibular lymph nodes are enlarged. You would look for potential infection sites in the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}