{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"625e41e8-8764-55fa-9f58-c555dc3fd740","task_key":"default--validation--13188","task_revision_id":"1","upstream_id":"13188","short_description":"HellaSwag validation 13188","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to hide a lip piercing from parents or bosses [title] Apply a silicone primer. [step] If your piercing has completely healed and you've removed the jewelry, dab silicone primer over the hole. A silicone primer should go on smoothly over the hole and help fill it in a little.\",\"endings\":\"['Primer makes the paint permanent and will not stick to the area. [title] Conceal the lip area with latex paint.', 'It should have been removed first; if it has not, the primer might cause it to come off too quickly. [title] Dab on a lip balm.', 'Primer will also help cover any blemishes or red areas on your face. [substeps] You can also put a little concealer over the primer to create the look of an even skin tone.', \\\"It should also completely settle in so that the jewelry is not sticking out. Wait about three minutes and you're done! [substeps] Silicone primer does not protect your jewelry from harmful chemicals, but it can prevent the area from becoming damaged and stained by harsh chemicals.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}