{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1493ace-d786-5b8c-8ee4-5ccbe443b0c0","task_key":"default--test--6241","task_revision_id":"1","upstream_id":"6241","short_description":"A 15-year-old girl is brought to the office by her mother because of abdominal…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Previous surgeries or hospitalizations', 'Exposure to sexually transmitted diseases', 'Menstrual history', 'History of urinary problems', 'Use of recreational drugs or alcohol', 'Recent travel history', 'History of food allergies']\",\"question\":\"A 15-year-old girl is brought to the office by her mother because of abdominal pain and constipation for the past several weeks. Her mother says, \\\"She is getting almost all A's in school and she is on the track team.\\\" You ask the patient about her diet and she responds, \\\"I'm kind of a picky eater.\\\" She requests a laxative to help with her constipation. She is 158 cm (5 ft 2 in) tall and weighs 43 kg (95 lb); BMI is 18 kg/m2 . Pulse is 65/min. Specific additional history should be obtained regarding which of the following? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}