{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d02bcef-93ed-530c-83f2-422fbfde21be","task_key":"default--validation--46375","task_revision_id":"1","upstream_id":"46375","short_description":"HellaSwag validation 46375","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a fur coat stop shedding [title] Keep your fur dry. [step] Don't wear your fur coat if you think it is going to rain or snow excessively. Getting a lot of water on your fur coat can damage it and can lead to shedding.\",\"endings\":\"['[substeps] You cannot always avoid getting a little rain or snow on your fur coat. However, if you are outside and it begins to rain or snow, take shelter.', 'Wear your fur coat whenever you exercise, go hiking or have any kind of activity, and always wash it in hot or cold water. [substeps] If your fur coat is made of cotton, it is more breathable.', \\\"If you're outside, the sweater or jacket you wear should be the basis of your fur coat. If rain is fine then by all means keep it dry.\\\", 'Avoid spending extra time brushing your coat if you feel like you need to keep it soft. [substeps] Consider investing in water-resistant brushes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}