{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc61c3b8-e6c0-5486-8efb-fda5fbcf1ad9","task_key":"default--test--9257","task_revision_id":"1","upstream_id":"9257","short_description":"By locating the north celestial pole (NCP) in the sky how can you determine your…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The altitude of the NCP is your angular distance from the North Pole.', 'The azimuth of the NCP is your angular distance from the equator.', 'The altitude of the NCP is your angular distance from the equator.', 'The altitude of the NCP is the same as your latitude.', 'The altitude of the NCP is the same as your longitude.', 'The azimuth of the NCP is the same as your latitude.', 'The azimuth of the NCP is the same as your longitude.', 'The azimuth of the NCP is your angular distance from the South Pole.', 'The altitude of the NCP is the angular distance from the South Pole.', 'The azimuth of the NCP is the angular distance from the North Pole.']\",\"question\":\"By locating the north celestial pole (NCP) in the sky how can you determine your latitude?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}