{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bac40e4e-2dac-54db-85f7-76818ea55ebc","task_key":"default--test--1148","task_revision_id":"1","upstream_id":"1148","short_description":"After her husband died in a hospital, a widow directed the hospital to send her…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the widow was never in any danger of bodily harm.', \\\"No, because the hospital's mistake was not intentional.\\\", \\\"Yes, because the negligent handling of the husband's body was especially likely to cause his widow serious emotional distress. \\\\x80\\\\x83\\\", 'Yes, because the hospital was responsible for the correct identification and handling of the body.', 'Yes, because hospitals are strictly liable if they do not properly dispose of corpses.', 'No, because the widow did not suffer any physical harm.', 'No, because the widow did not witness the cremation.', 'No, because the widow was not physically present at the time of the incident.', 'Yes, because the hospital should have double-checked the identity of the body before sending it for cremation.']\",\"question\":\"After her husband died in a hospital, a widow directed the hospital to send her husband's body to a funeral home for burial. The hospital negligently misidentified the husband's body and sent it to be cremated. When she was informed of the hospital's mistake, the widow suffered serious emotional distress. She has sued the hospital. Is the hospital likely to be held liable to the widow?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}