{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e3d3708d-5559-5699-8652-a07647283cdd","task_key":"default--test--1134","task_revision_id":"1","upstream_id":"1134","short_description":"A detective found a footprint from a left-foot shoe at a murder scene. The print…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the shoe print alone is insufficient evidence without additional forensic evidence.', 'Yes, because the brand and wear pattern match the footprint found at the scene.', \\\"Yes, as the shoe could potentially match the print found at the scene, indicating the suspect's involvement.\\\", 'Yes, because the shoe can be linked directly to the suspect.', 'No, because this shoe is irrelevant and will have no residue or other probative evidence relating to it.', \\\"No, because the right shoe cannot prove the suspect's presence at the crime scene.\\\", 'Yes, because it is appropriate circumstantial evidence of guilt.', 'No, because footprint evidence is highly speculative unless both shoes and shoe prints are offered and authenticated.', 'No, because the left shoe was not found and thus the evidence is incomplete.', 'Yes, because it is direct evidence of guilt.']\",\"question\":\"A detective found a footprint from a left-foot shoe at a murder scene. The print was preserved appropriately as evidence. It had distinctive tread marks and an unusual wear pattern on the sole. It also had a \\\"V\\\" mark on the heel bottom that indicates the brand was a Victory shoe. The detective, armed with a proper search warrant, searched the suspect's apartment, where he found a shoe to a right foot that of the same size, and with a similar wear pattern and the same \\\"V\\\" mark as the shoeprint found at the scene. The shoe for the left foot was not found but the shoe for the right foot was seized from the suspect's closet and offered as evidence at trial. Is this admissible evidence?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}