{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1b17086-ea13-5c10-8786-08204ab35a04","task_key":"default--validation--32124","task_revision_id":"1","upstream_id":"32124","short_description":"HellaSwag validation 32124","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply nail polish neatly [title] Remove old polish from your nails. [step] In order to produce a neat and even manicure, you must first remove any old polish from your nails. Acetone nail polish removers dry out your nail, and cuticle.\",\"endings\":\"['When removed and dried, paint your nails with moisturizing polish remover or foot scrub. [substeps] Acetone nail polish remover is brand new, so it should be easiest to apply when you have some time to spare.', \\\"Using the back of a spoon, dump the old polish into the small container you've left off when you're done. [substeps] When you're finished mixing the acetone, you should be able to no longer see your old polish to the right.\\\", \\\"Once your nail polish remover is dissolved, you can gently remove old polish from your nails that has dried to the point of flakes. [substeps] Don't worry if the area near the polish remover has dried to the point of flakes; many acetone nails are made with waterproof acetone.\\\", 'If possible, use a non-acetone nail polish remover over an acetone nail polish remover. [substeps] Place a cotton swab or ball over the opening of your nail polish remover.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}