{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27aa140e-92e2-5692-a710-3abe3b57b6eb","task_key":"all--test--27aa140e-92e2-5692-a710-3abe3b57b6eb","task_revision_id":"1","upstream_id":"","short_description":"To ensure that hatchling sea turtles orient to the sea rather than inland, which…","config":"all","split":"test","body":"{\"choices\":\"['Digging troughs in the sand from the nests to the sea', 'Minimizing artificial lighting in the area', 'Scattering preferred turtle foods between the nests and the sea', 'Building low fences on the sand dunes']\",\"question\":\"To ensure that hatchling sea turtles orient to the sea rather than inland, which of the following is the most environmentally sound practice around sea turtle nest sites?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}