{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fc99629-75aa-5380-9b74-75df031913c4","task_key":"default--test--6240","task_revision_id":"1","upstream_id":"6240","short_description":"A 15-year-old girl is brought to the office by her mother because of abdominal…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Menstrual history', 'History of food allergies', 'Recent travel history', 'Previous surgeries or hospitalizations', 'Exposure to sexually transmitted diseases', 'History of urinary problems', 'Use of recreational drugs or alcohol']\",\"question\":\"A 15-year-old girl is brought to the office by her mother because of abdominal pain and constipation for the past several weeks. Her mother says, \\\"She is getting almost all A's in school and she is on the track team.\\\" You ask the patient about her diet and she responds, \\\"I'm kind of a picky eater.\\\" She requests a laxative to help with her constipation. She is 158 cm (5 ft 2 in) tall and weighs 43 kg (95 lb); BMI is 18 kg/m2 . Pulse is 65/min. Specific additional history should be obtained regarding which of the following?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}