{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1387587-89bd-5358-8ee2-3b52acc10762","task_key":"default--validation--30374","task_revision_id":"1","upstream_id":"30374","short_description":"HellaSwag validation 30374","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply dark eye makeup [title] Apply a matte, light brown eyeshadow to the crease of your eyelid. [step] Use a soft, fluffy eyeshadow brush with longer bristles to do this. This will give you that nice, soft base color.\",\"endings\":\"['[title] Using a gray eyeshadow brush, brush a thin layer of dark eyeshadow over the eyelid. [step] Take a hold of the crease and move your brush down so it drops.', \\\"[substeps] To help the shadow extend past the crease, try misting the eyeshadow over the area of your eyelid that's already smudged. This will help the lighter shadow stay longer in a dark area.\\\", '[substeps] Apply a thin layer of eyeshadow at the crease, focusing mainly on the outer edge of your eyelid, and shifting to your outer crease to create your base. This will cover up wrinkles and make the eyes look less symmetrical.', '[substeps] If you have a darker skin tone, a light brown might be too light for you; opt for a shade that is 2 to 3 shades darker than your skin tone instead. [title] Blend the eyeshadow upwards towards your eyebrow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}