{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39acee6b-d1f3-593b-84a4-36afca5fbc58","task_key":"all--test--39acee6b-d1f3-593b-84a4-36afca5fbc58","task_revision_id":"1","upstream_id":"","short_description":"A key benefit of the STP process is:","config":"all","split":"test","body":"{\"choices\":\"['More effective and efficient matching of company resources to targeted market segments promising the greatest ROMI.', \\\"Enhancing a company's competitive position by providing direction and focus for marketing strategies.\\\", 'Examining and identifying growth opportunities in the market through the identification of new customers, growth segments, or new product uses.', 'All of the above.']\",\"question\":\" A key benefit of the STP process is:\",\"subject\":\"marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}