{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e465cad-214e-5c94-9129-8cb250662535","task_key":"default--validation--20682","task_revision_id":"1","upstream_id":"20682","short_description":"HellaSwag validation 20682","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean dyeable shoes [title] Read the care label. [step] Not all dyeable shoes will come with a care label, but many do. It is important to read the care label before attempting to clean the shoes.\",\"endings\":\"['Some brands, such as adidas and hollister, will ship shoes with care labels. Make sure to check the date and style of the shoes before attempting to clean them.', \\\"[substeps] Check the manufacturer's care label or the online retailer that carries the shoes, too. If you are concerned about damaging your shoes, call the manufacturer's sales assistant.\\\", 'The care label will contain product instructions for cleaning the shoes. For instance, if the shoes are sparkling clean but the toe area is plagued by dirt and grit, you may not be able to clean them.', \\\"If the care label instructs you to take them to a professional, it is best to do so if possible. [substeps] If your shoes don't come with a care label, you can do an online search for the type of shoes you have to see if there is any advice and/or instructions available.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}