{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e3546dd7-3fed-5147-87df-7140c594712c","task_key":"default--validation--18879","task_revision_id":"1","upstream_id":"18879","short_description":"HellaSwag validation 18879","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get perfect feet for summer [title] Remove any lingering polish or nail enamel. [step] Soak a cotton ball or pad in a small amount of polish remover. Rub the pad across your toe until the polish has dissolved and soaked into the cotton.\",\"endings\":\"['[substeps] For sensitive skin, use non-acetone polish remover. It may take a little longer to dissolve the polish, but it is gentler on skin and strips away less moisture.', 'Grab a cotton ball (preferably with a clear bristle point) and rub the cotton over your toes, all the way down your quick and hard toe. This will remove much of the polish from inside your toe.', 'Follow your toes with the cotton ball to check if polish has dried. Be sure to keep polishing the inside of your foot with the cotton swab.', 'This is the part of your foot that is the dry most often when exercising. [title] Apply lotion, body spray or a scented oil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}