{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdba762e-1c50-5f16-b492-2623bf990690","task_key":"default--test--1111","task_revision_id":"1","upstream_id":"1111","short_description":"A plaintiff filed a civil action asking for damages resulting from a tortious…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The objection will be sustained because the question is misleading.', 'The objection will be overruled because the question is relevant to the case.', 'The objection will be overruled because leading questions are allowed on cross examination.', 'The objection will be sustained because the question is argumentative.', 'The objection will be overruled because the question is not leading or misleading.', 'The objection will be sustained because the question is irrelevant.', 'The objection will be overruled because leading questions are permitted in this context.', 'The objection will be sustained because the question assumes facts not in evidence.', 'The objection will be sustained because the question is leading.', 'The objection will be overruled because the question is within the allowable scope of cross examination.']\",\"question\":\"A plaintiff filed a civil action asking for damages resulting from a tortious assault that the defendant inflicted on him. During trial, plaintiff's counsel asked an opposing witness on cross examination: \\\"At any time during the assault did anyone attempt to leave?\\\" Defense counsel objected to the question. How will the court rule on the objection?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}