{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70580bb9-8e67-582b-a5de-434bc977c965","task_key":"default--test--3721","task_revision_id":"1","upstream_id":"3721","short_description":"A proposed mechanism for a reaction is as follows: NO2 + F2 →NO2F + F Slow step.…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Fourth order', 'Second order', 'First order', 'Zero order', 'Third order in NO2 and zero order in F2', 'Second order in F2 and third order in NO2', 'First order in NO2 and second order in F2', 'Fifth order', 'Third order', 'Second order in NO2 and first order in F2']\",\"question\":\"A proposed mechanism for a reaction is as follows: NO2 + F2 →NO2F + F Slow step. F + NO2 → NO2F Fast step. What is the order of the overall reaction?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}