{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54181dcc-0ad7-58eb-845f-ba64614df95d","task_key":"default--validation--23208","task_revision_id":"1","upstream_id":"23208","short_description":"HellaSwag validation 23208","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to polish shoes [title] Choose your polish. [step] Shoe polishes are available in wax, cream and liquid forms. Waxes and creams are heavier and will feed the leather and protect the shoes from water damage.\",\"endings\":\"['You can buy polish at any shoe store, but it will generally be sold in a de-wax or opaque wax form. Choose specific polish brands and quantities of polishes for ease of application.', \\\"Wax flakes the lace or toenail while the polishes are held together. Choose one and you'll want to take both.\\\", 'Although different brands of polish are sold, allow one size at a time to change between jobs. You may even be able to get jobs with gloss polishes since these polishes tend to clean up quickly and should not be changed completely overnight.', 'Liquid polishes are good for a quick and easy shine. Shoe polishes are available in a variety of colors--you can buy specific shades to match the shoes you wish to polish, or you can buy a neutral polish which will work on a variety of shoe colors.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}