{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81a63270-4f2d-5121-b547-6184238f2516","task_key":"all--test--81a63270-4f2d-5121-b547-6184238f2516","task_revision_id":"1","upstream_id":"","short_description":"By locating the north celestial pole (NCP) in the sky how can you determine your…","config":"all","split":"test","body":"{\"choices\":\"['The azimuth of the NCP is the angular distance from the North Pole.', 'The azimuth of the NCP is the same as your latitude.', 'The altitude of the NCP is the same as your latitude.', 'The altitude of the NCP is your angular distance from the North Pole.']\",\"question\":\"By locating the north celestial pole (NCP) in the sky how can you determine your latitude?\",\"subject\":\"astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}