{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"043f96fb-d3e4-59f7-a31e-d33cd7ce15ce","task_key":"default--test--1817","task_revision_id":"1","upstream_id":"1817","short_description":"A shopper slipped and fell on a wet spot at the local mall, suffering injuries.…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"granted, because the housekeeper's testimony is hearsay not within any recognized exception.\\\", \\\"granted, because the housekeeper's testimony is based on the shopper's subjective self-assessment.\\\", \\\"denied, because the shopper's statement to her housekeeper would be admissible as an excited utterance.\\\", \\\"denied, because the housekeeper's testimony would be admissible as an adoptive admission.\\\", \\\"granted, because the housekeeper's testimony is not related to the case.\\\", \\\"denied, because the shopper's statement to her housekeeper would be admissible as a declaration of present bodily condition.\\\", \\\"denied, because the housekeeper's testimony would be admissible as a present sense impression.\\\", \\\"denied, because the housekeeper's testimony would be admissible as a statement for the purpose of medical treatment.\\\", \\\"granted, because the housekeeper's testimony is not based on her firsthand knowledge.\\\", \\\"granted, because the housekeeper's testimony would be self-serving.\\\"]\",\"question\":\"A shopper slipped and fell on a wet spot at the local mall, suffering injuries. The shopper has sued the mall owner for negligence. At trial, the shopper's housekeeper testified that the shopper was bedridden and unable to work for a month because of the sprained back she suffered when she fell at the mall. The housekeeper also testified that about one week after the shopper returned home from the hospital, the shopper told her, \\\"My back is really killing me. The pain is excruciating. It'll be a miracle if I can ever stand up again. \\\" The mall's counsel then moved to strike the housekeeper's testimony. The motion should be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}