{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29fdc5b4-cf40-5b31-ba88-a662b5f14d6a","task_key":"default--test--9807","task_revision_id":"1","upstream_id":"9807","short_description":"Does an ideal gas get hotter or colder when it expands according to the law pV^2…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['remains the same', 'none of these', 'hotter', 'the temperature oscillates', 'it gets colder, but only under low pressure', 'it first gets colder, then hotter', 'the temperature doubles', 'the change in temperature depends on the gas type', 'colder', 'it gets hotter, but only under high pressure']\",\"question\":\"Does an ideal gas get hotter or colder when it expands according to the law pV^2 = Const?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}