{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0e9d61f-fe64-5e98-ba70-80a8e8ca1469","task_key":"default--validation--50386","task_revision_id":"1","upstream_id":"50386","short_description":"HellaSwag validation 50386","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to create butterfly wing nail art [title] Prepare your nails. [step] Clean, trim, and file your nails. Push back your cuticles and apply some cuticle oil.\",\"endings\":\"['Lastly, wipe your nails down with some rubbing alcohol or nail polish remover. This will get rid of any oils that might prevent the polish from sticking.', 'Apply a coat of nail polish on your nail and move onto your next step. Press your cuticles towards your nails to prevent the manicure from breaking up or chipping.', 'Pat your cuticles back in with a pad or the tips of your fingers. [substeps] Make sure to file all rough edges into the shape of butterfly wings.', '[title] Choose a color that matches your skin tone. [step] Decide if you want your nail color to be muted or intense.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}