{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cca5b734-d4cf-5be4-82b8-856c415f1b3e","task_key":"default--validation--31976","task_revision_id":"1","upstream_id":"31976","short_description":"HellaSwag validation 31976","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to use makeup to make eyes and lips appear larger [title] Apply concealer. [step] Concealer in a shade slightly lighter than your skin tone illuminates the eye area, eliminates dark circles, and draws attention to dark lashes. Dab it underneath and around the eyes, and blend.\",\"endings\":\"['[title] Pick a neutral shade of eyeshadow that matches your skin tone to use as a base. [step] Apply it from your lash line to your brow bone.', '[substeps] Do not blend the concealer into the eyes or lips. Blending it into the eyes increases the shadow, but will not bring the eyes into focus.', '[substeps] Avoid using pale, translucent concealer, since it could play as a much more dramatic effect. Thinner concealers should still complement dark eye circles, and avoid green around the eyes, as these will always overpower dark circles.', \\\"Make sure the concealer doesn't overshadow your face in order to keep the eyes and lips from being discolored. Blend gently to avoid smudging eyes.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}