{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fdcc7aa0-9187-5d1c-b9e6-f2932bc1e77b","task_key":"default--validation--37398--variant-c7c3f8911249","task_revision_id":"1","upstream_id":"37398--variant-c7c3f8911249","short_description":"HellaSwag validation 37398","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a temporary tattoo with nail polish [title] Know the effect on your skin. [step] Many types of nail polish contain toxins that can be dangerous when left on your skin for a period of time. Some nail polishes are labeled as \\\" non-toxic, \\\" but it's found that these labels are occasionally inaccurate.\",\"endings\":\"['[substeps] In some places, in the u.s., chemical peels are prescribed. These peels help destrip the nail.', \\\"They may even be inaccurate or include a adverse effect. This is why it's a good idea to purchase nail polish for temporary tattoos as well, so you can avoid potentially permanent side effects.\\\", \\\"There are also nail polishes that are label free, but do not contain any toxins. Before you apply nail polish to your skin, it's best to check the ingredients for harmful toxins.\\\", '[title] Wear latex or cream nail polish. [step] Latex and cream polishes can protect your skin from the elements and usually have a nice finish, while cream polishes also have darker reds in them and are often more effective on darker skin tones.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}