{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"014d7026-b6f5-5a61-8825-934543cb6525","task_key":"default--validation--17654","task_revision_id":"1","upstream_id":"17654","short_description":"HellaSwag validation 17654","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make elastic ribbon hair ties [title] Buy some 5/8-inch (16-millimeter) fold over elastic. [step] You can sometimes find it labeled as \\\" elastic ribbon. \\\" you can buy it online and in fabric stores.\",\"endings\":\"['[title] Cut a few of the ends of the fabric off of the elastic. [step] Lay the elastic onto your hair and then wrap the top side over the elastic.', \\\"If you can't find elastic in any craft store, try making your own elastic hair tie. [substeps] Cut off the top of 2 2-inch strips of elastic from some scrap fabric or felt.\\\", '[title] Cut out a rectangular piece of foam to the measurements of the shape you want to make. [step] You can use your scissors to cut the foam down to a precise edge.', 'It is a smooth elastic with a crease down the center. It comes in all sorts of colors and patterns, including glitter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}