{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ed4db29-136c-5846-af83-3babc2ecdafd","task_key":"default--test--1879","task_revision_id":"1","upstream_id":"1879","short_description":"A college student and a man got into a fight at a bar while playing pooi. The…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible as a statement of the declarant’s then-existing state of mind.', 'inadmissible as a character evidence.', 'inadmissible as hearsay not within any recognized exception.', 'admissible as a prior consistent statement.', \\\"inadmissible as a statement that is not in the defendant's favor.\\\", 'inadmissible because the classmate was not an eyewitness to the fight.', 'inadmissible as self-serving.', 'admissible as part of the res gestae.', 'admissible as an admission.', 'admissible as a statement of personal or family history.']\",\"question\":\"A college student and a man got into a fight at a bar while playing pooi. The college student allegedly started the fight when he struck the man in the head with a pool cue after the man sank a game-winning shot. At the college student's trial for assault, the prosecution called one of the college student's classmates to testify. The classmate was not at the bar during the fight. The classmate offered to testify that the next day at class, he approached the college student and said, \\\"It was wrong of you to start a fight with the man over a game of pool,\\\" at which point, the college student bowed his head and walked away. The classmate's testimony is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}