{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6128a916-77ff-541f-9d8d-19ff4a85ab6f","task_key":"default--test--1110","task_revision_id":"1","upstream_id":"1110","short_description":"An insured business owner stored valuable property in an insured warehouse owned…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because a business owner has a constitutional right to present and admit character evidence.', 'No, because the business owner has a right to defend his character in court.', 'Yes, because the facts do not apply to any of the several exceptions available in a civil case.', 'Yes, because the trial court has discretion to exclude character evidence.', 'Yes, because character evidence is not relevant to the facts of the case.', 'No, because evidence of good character is always admissible in court.', 'Yes, because the insurance company has a right to accuse the owner without proof of their claim.', 'No, because Rule 404(b) permits the use of such evidence to show relevant prior bad acts.', 'Yes, because Rule 404(a) is firm in not allowing for character evidence in a civil trial.', 'No, because Rule 404(a) allows character evidence in cases of arson.']\",\"question\":\"An insured business owner stored valuable property in an insured warehouse owned by the business. During night time hours, when there were no employees in the warehouse, the building and its contents burned to the ground by a raging fire. The fire department ruled that the fire had an incendiary origin. The fire report, however, did not identify a perpetrator. The insurer responded by rejecting coverage and accusing the business owner of starting the fire. The business owner sued the insurance company in federal court for the policy proceeds to cover its property damages claim. At the trial on the business owner's civil claim for insurance coverage, the owner attempted to put several persons on the stand to testify to the owner's good character, which was submitted as proof that the owner did not commit the arson. The trial court ruled that the character evidence would be excluded because it was inadmissible character testimony under Fed.R.Evid. 404(a). On appeal, will the fedeal appellate court likely affim the trial court's order?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}