{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4912395-7a7f-5530-a202-4db8a8de0e35","task_key":"default--validation--5465","task_revision_id":"1","upstream_id":"5465","short_description":"HellaSwag validation 5465","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove glitter nail polish [title] Gather your supplies. [step] Get some nail polish remover, a cotton ball, and aluminum foil. If you don't have aluminum foil, a rubber band or hair elastic will do the trick just as nicely.\",\"endings\":\"['[title] Put a small amount of the mixture on the cotton ball and squirt the solution onto the dried nail polish. [step] Save any liquid look by placing it on the area you want to clean.', 'Before washing your nails, try rubbing vaseline onto your fingers. You may also try soap from your bathroom or a toothbrush if you are a leave-in hand-care professional.', 'Work the remover into your nail without rubbing. [title] Place a cotton ball in a dish of nail polish remover.', 'What you really want is something that will hold the cotton ball in place against your fingernail. [title] Soak a cotton ball in acetone-based nail polish remover.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}