{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66e6e9ed-092e-5302-8eb7-d3a5ec123a0b","task_key":"default--test--6445","task_revision_id":"1","upstream_id":"6445","short_description":"A 44-year-old female presents to the office for evaluation of a lump on her neck…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Computed tomography (CT) scan', 'fine-needle aspiration', 'Magnetic resonance imaging (MRI)', 'a nuclear thyroid scan', 'Thyroid biopsy', 'Endoscopic examination of the throat', 'an iodine131\\\\xa0scan', 'Positron emission tomography (PET) scan', 'ultrasonography of the thyroid gland', 'Lymph node biopsy']\",\"question\":\"A 44-year-old female presents to the office for evaluation of a lump on her neck that she noted 1 week ago. She denies any tenderness, fever, weight loss, or fatigue. Physical examination reveals a 2-cm freely movable mass in the lower left lobe of the thyroid. In addition to thyroid-stimulating hormone and free thyroxine levels, the most appropriate initial method to investigate this lesion is \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}