{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65bb55ea-170c-56d7-808f-34db9991cd07","task_key":"default--validation--28133","task_revision_id":"1","upstream_id":"28133","short_description":"HellaSwag validation 28133","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear purple eyeshadow [title] Put eye primer or concealer on eyelid using fingers or a brush. [step] A primer really helps eyeshadow to blend better and stay longer. When applying make sure to fully blend it out.\",\"endings\":\"['[title] Line your eyelid using a black liner that matches your natural eye color. [title] Use this box as your base if you have black eye makeup primer on your eyelids.', 'After you have applied your primer you need to wrap the eyeshadow around your eyelid, which should be the only part of your eyelid that is covered. Depending on your eye color, cover only the eyeshadow with your primer and concealer.', '[title] If using a concealer to prime, put a light eyeshadow over it to help blending later. [step] A light seashell color works best.', '[title] Prise your eyelid with an eyeshadow brush or sponge. [step] Gently apply primer or eyeshadow to the inside corner of your eye.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}