{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44aa5f16-720f-56f7-9751-f414d7111e29","task_key":"default--validation--40153","task_revision_id":"1","upstream_id":"40153","short_description":"HellaSwag validation 40153","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to wash clothes by hand [title] Pick a mild detergent for delicate clothes. [step] Mild detergent is fine to use for most clothing items as long as they are not made of delicate fabrics like silk, lace, wool, or fine-knit material. Look for liquid detergent that is made for delicate clothing.\",\"endings\":\"[\\\"Plain, unscented detergent won't dry out delicate fabrics like silk and muslin, though. For fabrics like cotton, white, and wool, you can use a stronger detergent if they are delicate.\\\", 'Hypoallergenic detergent will not work on delicate clothes, as it will slow down the drying time of delicate items. [substeps] Use a detergent that is formulated for delicate fabrics, such as dish washing detergent, gentle detergent, or shopping detergent.', 'A detergent formulated for fabrics like cotton, linen, and leather will produce the best results. [substeps] Alternatively, look for a detergent made specifically for delicate delicate fabrics.', \\\"Brands like woolite or soak are good options. [substeps] In a pinch, a regular detergent like tide or gain will work for delicate clothing that's not made of lace, silk, or wool.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}