{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b8481e4-4388-5e62-accf-f05c3c298667","task_key":"all--test--5b8481e4-4388-5e62-accf-f05c3c298667","task_revision_id":"1","upstream_id":"","short_description":"Which of the following statements is false?","config":"all","split":"test","body":"{\"choices\":\"['Ammonia is produced in repeated high intensity exercise.', 'Muscle lactate accumulation does not begin until at least 5 seconds of intermittent muscle contractions have taken place.', 'Muscle phosphocreatine depletion begins in the first few seconds of high intensity exercise.', 'With an increasing number of repeated sprints the rate of accumulation of lactate in muscle declines.']\",\"question\":\"Which of the following statements is false?\",\"subject\":\"college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}