{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b4cbfaf-cf3c-54cd-afa9-be34cd3ed7da","task_key":"all--test--9b4cbfaf-cf3c-54cd-afa9-be34cd3ed7da","task_revision_id":"1","upstream_id":"","short_description":"Karen is a college student working on developing a stronger sense of self-esteem…","config":"all","split":"test","body":"{\"choices\":\"['Seek positive feedback from friends.', 'Put in daily practice on the tasks she wishes to improve on.', 'Find others her age and ability who excel at tasks she is interested in.', 'Avoid potential pitfalls by withholding from tasks she is not proficient in.']\",\"question\":\"Karen is a college student working on developing a stronger sense of self-esteem and self-efficacy with her therapist. She has noticed a great change in her ability to handle situations after 3 months of therapy. Which of the following would NOT be a strategy that her therapist would ask her to employ to raise her sense of self-efficacy?\",\"subject\":\"college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}