{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6c84645-160c-51d1-9c44-c56dbf0778bd","task_key":"default--test--9253","task_revision_id":"1","upstream_id":"9253","short_description":"From laboratory measurements we know that a particular spectral line formed by…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"[\\\"The star's brightness is fluctuating.\\\", \\\"The star's size is increasing.\\\", \\\"The star's size is decreasing.\\\", \\\"The star's temperature is constant.\\\", 'The star is getting hotter.', 'The star is not moving.', 'The star is rotating on its axis.', 'The star is getting colder.', 'The star is moving away from us.', 'The star is moving toward us.']\",\"question\":\"From laboratory measurements we know that a particular spectral line formed by hydrogen appears at a wavelength of 486.1 nanometers (nm). The spectrum of a particular star shows the same hydrogen line appearing at a wavelength of 485.9 nm. What can we conclude?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}