{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3eedd3e6-7b84-5e8d-824b-e5c8f8003fcb","task_key":"all--test--3eedd3e6-7b84-5e8d-824b-e5c8f8003fcb","task_revision_id":"1","upstream_id":"","short_description":"A husband and wife were shopping at a local department store when the wife saw a…","config":"all","split":"test","body":"{\"choices\":\"['win, because the statement was overheard byanother customer.', \\\"win, because the clerk's conduct was extreme and outrageous.\\\", 'lose, because the clerk was merely statingan opinion.', 'lose, because the wife only suffered hurt feelings.']\",\"question\":\"A husband and wife were shopping at a local department store when the wife saw a dress that she especially liked. The wife, who weighed more than 300 pounds, was unable to find a size large enough to fit her. She then saw a store clerk and asked if the store carried the dress in her size. The clerk looked at the wife and said, \\\"You look like a hippopotamus, and I'm sorry, but we don't carry this dress in the hippo size. \\\" Another customer in the store overheard the comment and began to laugh. The wife became very upset and hurried out of the store. If the wife asserts a tort action against the department store based upon the clerk's actions, the wife will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}