{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8658fd09-d105-505c-a4a0-ec5bed6fa834","task_key":"default--test--1619","task_revision_id":"1","upstream_id":"1619","short_description":"A teenager shot and killed a man she said tried to attack her as she was walking…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['inadmissible, because it could bias the jury against the victim.', 'inadmissible, because the evidence is being used to prove propensity.', \\\"admissible, as it could be used to establish the teenager's fear of the man.\\\", \\\"inadmissible, because it violates the Dead Man's Statute.\\\", \\\"admissible, because it proves the man's disposition to commit rape.\\\", \\\"admissible, because it supports the teenager's claim of self-defense.\\\", \\\"admissible, because the victim's prior convictions would be relevant on the issue of the defendant's self-defense.\\\", \\\"inadmissible, as it breaches the victim's right to a fair trial.\\\", 'inadmissible, as it is irrelevant to the current case.', \\\"admissible, because it provides a motive for the teenager's actions.\\\"]\",\"question\":\"A teenager shot and killed a man she said tried to attack her as she was walking home from an all night market. The teenager, 18 years old and 6 feet 3 inches tall, said a very short man placed a knife against her throat and unzipped her dress as she was walking down the street. She pulled a gun from her purse and shot him. The man died of a chest wound. Police said the man, who was 5 feet 5 inches tall and weighed 130 pounds, was an ex-felon, twice convicted of attempted rape. The teenager is charged with the murder of the man, as a result of the above described incident. At trial, the teenager wishes to offer evidence of the fact that the man was previously convicted of attempted rape. The evidence would be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}