{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"774f9826-44c9-59f0-a723-d102ffead2cf","task_key":"default--test--1758","task_revision_id":"1","upstream_id":"1758","short_description":"Point out the most accurate response : According to Austin, law has the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Command, sovereign and legal remedy', 'Command, sovereign and authority', 'Command, sovereign and legal enforcement', 'Command, sovereign and legal responsibility', 'Command, sovereign and obedience by subject', 'Command, sovereign and enforceability', 'Command, sovereign and sanction', 'Command, sovereign and compliance']\",\"question\":\"Point out the most accurate response : According to Austin, law has the following attributes :\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}