{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39a41ddd-ffeb-5542-8ae1-ec520b972ae2","task_key":"default--validation--1886","task_revision_id":"1","upstream_id":"1886","short_description":"HellaSwag validation 1886","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply neulash [title] Wash off eye makeup if you're wearing any. [step] Saturate a cotton pad with eye makeup remover, and gently run it over the top of your eyelashes. Then, take the pad and gently wipe in a downward motion to remove any makeup on your lower set of lashes.\",\"endings\":\"[\\\"[substeps] You don't need to do this for the rest of your lashes. Because neulash makeup often looks washed out, it can make your eyes appear to have blackheads or pimples.\\\", \\\"[substeps] If you don't have eye makeup remover, you can use baby shampoo as an alternative. [title] Make sure your eyelashes are completely dry before applying neulash.\\\", '[title] Grab the edge of a silk or chiffon pillowcase. [step] Create a hand-drawn pattern by placing a scarf along a section of the pillowcase.', 'Not all eye makeup removers work on your eyes. [substeps] Use a soft eye makeup remover if you have weak eyes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}