{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc7011c7-ed7f-5b9d-8ac3-9e17b34d4416","task_key":"default--test--1084","task_revision_id":"1","upstream_id":"1084","short_description":"A college student owns an expensive sports car. His friend called him up one…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"admissible as evidence of the friend's driving skill.\\\", \\\"inadmissible, because it's irrelevant to the case.\\\", 'admissible as a present sense impression.', \\\"inadmissible, because it's hearsay.\\\", \\\"inadmissible, because it's opinion testimony.\\\", 'admissible as a past recollection recorded.', 'admissible as character evidence.', 'admissible as evidence of habit.', 'inadmissible, because she is biased.', \\\"inadmissible, because it goes to the college student's character.\\\"]\",\"question\":\"A college student owns an expensive sports car. His friend called him up one afternoon and asked to borrow his car for a date he had that night. The college student generously lent the car to his friend. On his way home from the date, basking in the memories of a beautiful evening, the friend lost control of the car and hit a pedestrian. As a result of the accident, and after learning that the friend had no money and no insurance, the pedestrian brought suit against the college student to recover for his injuries. He asserted a claim against the college student for negligent entrustment of a vehicle. During his case, the pedestrian introduced evidence that the friend had been in several previous accidents. In his defense, the college student offered the testimony of his girlfriend that the college student frequently lent his car to his friend. The girlfriend further testified that the college student gave his friend permission to use his car only after determining that the friend was a careful and attentive driver. The girlfriend's testimony is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}