{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"866ed6fd-27ea-55f4-bb8c-39e09db73d22","task_key":"default--validation--5244","task_revision_id":"1","upstream_id":"5244","short_description":"HellaSwag validation 5244","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make loose eyeshadows and pigments stick [title] Prep the eye area with an eyelid primer. [substeps] Use a small amount for a thin layer. Gently pat the product in with your finger, brush (made specifically for emollient products), or a sponge.\",\"endings\":\"['This medium primer is specifically designed for making the type and type of eyeshadow you want to use. You can use every color you want, but black, brown, and blue work best.', \\\"[title] Perfect the contours of your eye. [step] Pencil in and out (this will give the illusion of depth) so the highlight doesn't go astray.\\\", 'Apply the primer sparingly to the eye area, as this will help the product penetrate the eye more effectively. Apply eyeshadow in 1 line; blending in with a flat brush to achieve a dollop.', '[title] Next we are going to add a creamy second \\\" base \\\". [step] This, unlike the first product we used, does not \\\" dry \\\".']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}