{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"266ee32c-55be-5737-80dd-ebbc6fc494c2","task_key":"default--test--1714","task_revision_id":"1","upstream_id":"1714","short_description":"A defendant was charged with attempting to possess and distribute narcotics. The…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"reversed, because the prosecutor's remarks constituted an unfair personal attack on the defendant.\\\", \\\"reversed, because the prosecutor's remarks constituted improper comment about the defendant's right to testify.\\\", \\\"affirmed, because it is immaterial; the probative value of the defendant's failure to testify is outweighed by the danger of unfair prejudice.\\\", \\\"affirmed, because the defendant had the opportunity to refute the undercover agent's testimony but did not.\\\", \\\"reversed, because the prosecutor's remarks violated the defendant's right to a fair trial.\\\", \\\"reversed, because the prosecutor's remarks were not proper rebuttal to defense attacks on the undercover agent's credibility.\\\", \\\"reversed, because the prosecutor's remarks unfairly influenced the jury's decision.\\\", \\\"affirmed, because the prosecutor's remarks were a fair response to the defense's attacks on the undercover agent's credibility.\\\", 'affirmed, because the prosecutor was merely summarizing the evidence presented in court.', 'affirmed, because the prosecutor had the right to express the strength of the evidence he had pursued.']\",\"question\":\"A defendant was charged with attempting to possess and distribute narcotics. The defendant was arrested after allegedly trying to purchase several kilos of heroin from an undercover government agent. At trial, the undercover agent testified about the defendant's efforts to arrange the drug transaction. The defendant's defense, on the other hand, consisted entirely in attacking the undercover agent's credibility. During his summation, the prosecuting attorney, over defendant's objections, stated that he had produced the only witness who was willing to take the stand and testify to what occurred at the meeting when the undercover agent arrested the defendant. Based on the undercover agent's uncontradicted testimony, the prosecuting attorney urged the jury to convict the defendant. The defendant was convicted and, on appeal, argued that the judge erred in permitting the prosecutor's statement. The defendant's conviction should be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}