{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a47bcd4c-53ac-5db6-82b9-9daec734af84","task_key":"default--validation--5926","task_revision_id":"1","upstream_id":"5926","short_description":"HellaSwag validation 5926","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to give your feet a makeover [title] Wash your feet. [step] Sounds simple enough, right? Wrong. Standing in soapy water doesn't count as washing your feet.\",\"endings\":\"[\\\"Removing bacteria from the feet is one thing; it's entirely another. Use a whole host of cleansers that are gentle, even at head to toe.\\\", 'Go for 1-2 whole baths a day to get rid of soap scum and dirt. Showering will reduce calluses, leading to slimmer, healthier foot feet.', 'Or if you have a foot infection-go for a more appealing option. [substeps] For example, if you have a stained toe, you could use homemade cleaner in place of soap.', 'Sit on the edge of your tub and give them a good wash, rubbing with soap and probably a brush if they are very dirty. Brush your toenails too, until they are clean.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}