{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"53fa3e4d-bab3-5e6a-8533-ab3e36b56897","task_key":"default--test--1399","task_revision_id":"1","upstream_id":"1399","short_description":"A plaintiff sued a trucking company for injuries allegedly suffered when a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Color pictures of a plastic surgeon operating on the plaintiff showing the incision and bloody surgical tools.', \\\"Testimony from the trucking company's insurance company about their willingness to settle the case.\\\", 'Testimony from a mechanic who inspected the truck after the accident and found it to be in perfect condition.', 'Testimony of a surprise witness to the accident when the court is willing to grant the trucking company a continuance to prepare for crossexamination.', \\\"Testimony from the plaintiff's doctor about her previous health conditions.\\\", \\\"Testimony from the plaintiff's boss concerning the amount of time she has missed work, when payroll records had already been admitted as evidence of the plaintiff's lost wages.\\\", \\\"Testimony from the plaintiff's mother about her daughter's pain and suffering.\\\", 'Testimony concerning subsequent repairs to the truck paid for by the trucking company where they have stipulated to ownership.', \\\"Testimony from the truck driver's spouse concerning the driver's habit of speeding.\\\", \\\"Photographs of the plaintiff's car before the accident, showing it in a damaged state.\\\"]\",\"question\":\"A plaintiff sued a trucking company for injuries allegedly suffered when a speeding truck jackknifed and struck her car. Which of the following pieces of evidence, if offered by the plaintiff, is most likely to be admitted by the court? \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}