{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b14ad8a8-c06d-5bc3-8576-7db8521223fd","task_key":"default--test--6146","task_revision_id":"1","upstream_id":"6146","short_description":"Which of the following is true regarding the complications of a large goitre?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['It causes tinnitus due to compression of the auditory canal', 'It causes severe headaches due to increased pressure in the neck area', 'It causes shortness of breath due to compression of the lungs', 'It causes hoarseness due to compression of the trachea', 'It causes a more florid thyrotoxicosis than a smaller goitre', 'It can cause hyperthyroidism due to overproduction of thyroid hormones', 'It causes stridor due to compression of the oesophagus', 'It leads to a decreased production of thyroid hormones causing hypothyroidism', 'It increases the risk of thyroid cancer', 'Dysphagia is an indication for surgery']\",\"question\":\"Which of the following is true regarding the complications of a large goitre?\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}