{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"693ae06e-606f-5dc6-b9a2-5ad8761f389b","task_key":"default--validation--31086","task_revision_id":"1","upstream_id":"31086","short_description":"HellaSwag validation 31086","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for a leather jacket [title] Make your leather water resistant. [step] There are many \\\" leather protector \\\" or \\\" leather waterproofing \\\" products, but read the label carefully before buying. A silicone polymer spray, or an acrylic copolymer spray, should preserve your leather's appearance and shine.\",\"endings\":\"['This will help prevent eczema from forming or off spotting. The oil will improve the pores, and reduce scars from absorbing oil.', 'Grease-based or wax-based products are more protective, but not recommended for jackets due to their potential effect on color, longevity, shine, and smell. Check the label on your product to find out how long the water resistance lasts.', 'After you apply your oil, rinse your leather with lukewarm water. If you use a spray bottle, the bottom of the bottle must be clean, and there must also be no running water when you spray your leather when you apply the oil.', '[title] Wrap your jacket in a towel to squeeze the excess water out. [step] Secure the towel with a rubber band and apply pressure as you remove the towel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}