{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"186e45ce-ffb9-5a0c-bc5c-013629183bc4","task_key":"default--validation--17104","task_revision_id":"1","upstream_id":"17104","short_description":"HellaSwag validation 17104","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean a beauty blender [title] Prepare a dish of warm, soapy water. [step] Fill a small dish with warm water, then squeeze a dollop of gentle hand soap or shampoo into it. Mix lightly until suds form on the surface of the water.\",\"endings\":\"['Soap tends to leave a film on the blades of the blender, so use cool water if you can. The hot water will soften the blades, making them easier to clean.', '[substeps] The dish of water can use all water or just warm, soapy water; some diluted soap should work just fine; any fresh, unused water will do. Soap with natural herbs or citrus will also work as well.', 'Dish soap is generally not recommended, as it can irritate your skin when you need to use the sponge later. [substeps] Baby shampoo and \\\" gentle formula \\\" organic shampoos work especially well, but most soaps that are safe for use on your skin or hair can be used here.', \\\"Pour the suds into a small-sized bowl. Don't fill the dish more than 3/4 full, otherwise the water will seep out and the blender will drown.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}