{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd942698-a952-5da2-a007-7ec4d9776eb6","task_key":"default--test--3438","task_revision_id":"1","upstream_id":"3438","short_description":"The reaction A + B → C is catalyzed by enzyme K. If the reaction is in…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['removing some of reactant C', 'adding a different enzyme', 'decreasing the temperature of the system', 'adding more enzyme K', 'removing some of reactant A', 'increasing the temperature of the system', 'increasing the pressure of the system', 'removing some of enzyme K']\",\"question\":\"The reaction A + B → C is catalyzed by enzyme K. If the reaction is in equilibrium, which of the following would allow more product C to be produced?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}