{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4187db6b-a6de-5195-b0d7-cf7a5b2d06c5","task_key":"default--test--6753","task_revision_id":"1","upstream_id":"6753","short_description":"A 15-year-old girl comes to the emergency department because, she says,…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Obtain written consent from her 28-year-old cousin', 'Obtain written consent from at least one of her parents', 'Obtain a court order permitting evaluation', 'Obtain the written consent of at least two licensed physicians', \\\"Obtain the girl's consent in the presence of a witness\\\", \\\"Obtain written consent from the child's father\\\", 'Obtain verbal consent from at least one of her parents', 'Wait until the girl turns 16 to perform the evaluation', \\\"Accept the girl's consent as sufficient\\\", 'Obtain written consent from a social worker']\",\"question\":\"A 15-year-old girl comes to the emergency department because, she says, \\\"something has been sticking out of my bottom since I had a bowel movement this morning.\\\" She has not had previous episodes, although for more than 1 year she has had occasional difficulty passing stools. She is not in pain but is afraid to move her bowels for fear that the problem will worsen. She tells you that she moved away from home more than a year ago and that her parents contribute nothing to her support. She has a 6-month-old child and lives with a 28-year-old female cousin. She has never been married and does not work or attend school. She has no other symptoms. In order to follow the correct procedure for treating a minor, which of the following is the most appropriate step prior to evaluating this patient's rectal problem?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}