{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"589af2c8-53b8-591a-a7f6-8dff2b42920b","task_key":"default--test--3810","task_revision_id":"1","upstream_id":"3810","short_description":"When a solid melts, the entropy change and enthalpy changes expected are","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Positive enthalpy change and negative entropy change', 'negative entropy change and a negative enthalpy change', 'No change in entropy and positive enthalpy change', 'No change in either entropy or enthalpy', 'Negative enthalpy change and no change in entropy', 'Negative enthalpy change and negative entropy change', 'Positive entropy change and no change in enthalpy', 'positive enthalpy change and positive entropy change', 'negative entropy change and positive enthalpy change', 'negative enthalpy change and positive entropy change']\",\"question\":\"When a solid melts, the entropy change and enthalpy changes expected are\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}