{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4459210-79f0-56d0-a85d-8427b636c582","task_key":"default--validation--18272","task_revision_id":"1","upstream_id":"18272","short_description":"HellaSwag validation 18272","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a dip powder manicure [title] Use polish remover as well as a cuticle pusher. [step] If you have polish on your nails, remove it with non-acetone nail polish remover on a lint-free pad. Use a cuticle pusher to gently nudge your cuticles slightly back.\",\"endings\":\"['[substeps] Only use polish remover or a cuticle pusher if you have dirty hands. [title] Write on your nails with the cuticle pusher.', \\\"Do not push them back forcefully so you don't cause pain to your fingers or toenails. However, press them firmly with your fingers.\\\", 'Work the pusher in a circular motion to stimulate blood flow. [title] Apply a leave-in conditioner after misting your cuticles.', 'Gently remove any excess cuticle sticking up with a cuticle trimmer or scraper. [substeps] This will allow new nail growth to show so that your manicure will last longer before needing a fill.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}