{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02ec8a26-0120-5526-8a4d-250fe88948ce","task_key":"default--test--9942","task_revision_id":"1","upstream_id":"9942","short_description":"The pressure at the bottom of a pond does NOT depend on the","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['salinity of the water.', 'proximity to the equator.', 'presence of fish in the pond.', 'type of pond vegetation.', 'acceleration due to gravity.', 'surface area of the pond.', 'depth of the pond.', 'color of the water.', 'water density.', 'temperature of the water.']\",\"question\":\"The pressure at the bottom of a pond does NOT depend on the\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}