{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d151f208-9f11-5272-b3fe-8c56ca60c228","task_key":"all--test--d151f208-9f11-5272-b3fe-8c56ca60c228","task_revision_id":"1","upstream_id":"","short_description":"How do gun shows and international arms fairs contribute to illicit arms…","config":"all","split":"test","body":"{\"choices\":\"['Although the subject of vocal opposition by campaigners, gun shows employ strict legal restrictions on participants, ensuring that those with criminal records are not able to purchase weapons.', 'Gun shows are regulated to ensure that weapons purchased legally cannot be resold on the black or grey markets.', 'Gun shows and arms fairs do not contribute in a significant way to the trade in illicit arms.', 'Gun shows and arms fairs often do not require participants to undergo security checks before purchasing weapons, and encourage an environment where more attention is paid to price tags than details about where weapons will be distributed or used.']\",\"question\":\"How do gun shows and international arms fairs contribute to illicit arms markets?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}