{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"569e0ab1-090a-5b1a-8839-9a3a96b862d2","task_key":"default--validation--28717","task_revision_id":"1","upstream_id":"28717","short_description":"HellaSwag validation 28717","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to get a red wine stain out of a white linen shirt [title] Remove your shirt. [step] Act quickly. Immediately after the stain happens, go change into something else and take off your linen shirt.\",\"endings\":\"['If you notice the stain, you can apply an stain remover, if need be. This you can spray anywhere on your shirt, including the hem and around the collar and armpits.', 'If you have a fresh white linen shirt that you feel confident wearing, you can clean it using a scouring powder and hand wash it as usual. Just be careful not to soak or otherwise otherwise damage your linen shirt.', 'Make sure that as you are taking off the shirt, you do not touch any other part of the shirt to the stained section. This could spread the stain to other parts of the shirt.', '[title] Soak the stained area in hydrogen peroxide and water. [step] Hydrogen peroxide contains a disinfecting agent that stops bacteria from getting into stained shirt fabrics.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}