{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c2bb6d57-c2f7-522a-bf5c-cba4c415317e","task_key":"default--test--937","task_revision_id":"1","upstream_id":"937","short_description":"A man kidnapped a victim in State A and transported the victim across the state…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the man committed two separate acts of kidnapping, one in each state.', 'Yes, because the man was already tried and convicted for the same crime in State A.', 'Yes, because the double jeopardy protection applies to both state and federal laws.', \\\"No, because the man's conduct violated the laws of two different states, and each has the right to enforce its laws. right to enforce its laws.\\\", 'No, because the double jeopardy protection only applies to federal law.', 'Yes, because the double jeopardy protection applies even when laws of different states are identical.', \\\"No, because the man's sentence in State A was deemed unduly lenient in State B.\\\", 'Yes, because the government is collaterally estopped under the double jeopardy protection from prosecuting a defendant a second time for the same conduct that led to an earlier conviction.', 'Yes, because the double jeopardy protection prohibits a second prosecution following a conviction for the same offense.', 'No, because the double jeopardy protection only prohibits a second prosecution following an acquittal for the same offense.']\",\"question\":\"A man kidnapped a victim in State A and transported the victim across the state border into State B. These actions violated the kidnapping laws of both states, which are identical. A jury convicted the man in a State A court, but the trial judge gave what prosecutors in State B deemed an unduly lenient sentence. The state prosecutor in State B then commenced a kidnapping case against the man for violating State B's kidnapping statute. The man's lawyer has filed a motion in State B to dismiss the charge based on the double jeopardy protection against a second prosecution for the man's single act of kidnapping. Should the court grant the motion to dismiss?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}