{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d84c205b-a7d4-528f-95e8-17c1dced6720","task_key":"default--validation--21727","task_revision_id":"1","upstream_id":"21727","short_description":"HellaSwag validation 21727","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to create star nail art [title] Gather your materials. [step] Painting simple stars on your nails is much easier than you might think. You just need the right tools to do it.\",\"endings\":\"['Wood work, scissors, a nail cuticle pusher, a stain remover, and a cotton swab or a toothbrush. [substeps] Or, try to find some woodland sticks or twigs with interesting shapes or thin stems.', \\\"You can purchase paintbrushes at craft stores, or you can ask an art store or art supply store if they have them. [substeps] You'll need a sharpie palette, a spoon or palette knife, plates, brushes, and wooden dowels (any type will work).\\\", 'You will need : [substeps] Nail polish in the color (s) of your choice fine tipped nail brush or paintbrush clear topcoat [title] Paint your nails and let them dry. [step] Start by painting your nails in the color (s) of your choice.', 'You can purchase a variety of craft tools from a craft store for a relatively cheap price at around $6 or less. [substeps] The best tools to use for this project is a hammer, mirror, paint brush, glass and a water bottle (make sure your bottle is clean and can be reused).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}