{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe424475-3d58-5bb7-9de6-9786c7d57a0f","task_key":"default--validation--19615","task_revision_id":"1","upstream_id":"19615","short_description":"HellaSwag validation 19615","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do winged eyeliner [title] Prep your eyelid with concealer or primer. [step] Use a light pencil eyeliner to draw a thin line as close to your upper lash line as possible. The line should be as thin as possible.\",\"endings\":\"['Place your ring finger across the top lash line and set the outer corners of your winged eyeliner line to the outer corner of your upper lash line. Drag the corner straight up towards your eye.', 'Cover a large area by dabbing concealer/primer on it using the back of a makeup sponge. [substeps] After you have covered the entire eyelid, apply highlighter to the thicker, longer lashlines.', 'This acts as a base for your winged eyeliner. [substeps] Start from the inner corner of your eye and work toward the outer corner.', 'Look for it in the mirror so you can get a clear, neat design. [substeps] Alternatively, apply the eyeliner one or two shades lower so that it traces all the way over the lash line.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}