{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7445e3e1-27a7-5b7a-9df8-1eb67e9f3d0a","task_key":"default--validation--30058","task_revision_id":"1","upstream_id":"30058","short_description":"HellaSwag validation 30058","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to select shoes to wear with an outfit [title] Choose shoe colors that match your clothes instead of colors that compete with your clothes. [substeps] Wear simple black heels or flats when wearing a dress with a bright, bold pattern. If you were to wear a more complex shoe, it would be too overwhelming in the opinion of some.\",\"endings\":\"['Wear a long, flowing skirt or floor length dress to a very casual event like a party or easter fest. Wear a strappy top or thin leggings if a more feminine style can be worn with this article of clothing.', \\\"Limit your visibility that way and you'll be drawing more attention to your outfit as a result. Wear socks with long pant legs to hide your feet from the rest of the outfit.\\\", 'Instead, wear a thin pink or magenta knee-high shoe to match your dress or suit pants. [title] Order a few sneakers in a shade of pink, or heels to match your outfit.', \\\"Obviously, you can wear whatever shoes you like provided there is no dress code to obey or health and safety issues to consider. Consider neutral or' nude' heels or flats if you have on a sparkly evening top.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}