{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a24a365-cdca-59df-8f1c-a6a0abc1657e","task_key":"default--validation--3555","task_revision_id":"1","upstream_id":"3555","short_description":"HellaSwag validation 3555","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to take an epsom salt bath [title] Draw a warm bath. [step] Very hot water feels great, but warm water is best for your skin. Fill the bath with warm water.\",\"endings\":\"['Fill it enough so that you submerge most of your body in the water. [title] Add 2 cups of epsom salt to the water.', \\\"Take a towel and cover your head and you're already warm, but this only takes a few minutes. [substeps] Don't fill the water too deep, as it could evaporate or burn your skin.\\\", 'Epsom salt is not harmful to your skin, and will help replenish water lost throughout the day. You can also use a cup of lukewarm water.', \\\"If your long hair is wet, you're more likely to dry it out. Salt baths are designed to hydrate very dry hair, so you don't want to soak wet hair in hot water.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}