{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c485377-d903-59b5-9061-97521fddc9f1","task_key":"default--validation--27236","task_revision_id":"1","upstream_id":"27236","short_description":"HellaSwag validation 27236","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to store false eyelashes [title] Remove your false eyelashes. [step] Always remove your false lashes with your fingertips. Never use a tool such as tweezers or even your fingernails to remove them.\",\"endings\":\"['Otherwise, you may poke yourself in the eye. Just use the pads of your fingertips to gently peel off your false eyelashes.', '[title] Rub a petroleum based cleanser over your lashes, lips, and eyelids before you apply false eyelashes. [step] Petroleum products can reduce eyelash redness and treat excessive hair damage.', \\\"If someone in the store has been looking at your eyelashes, you can remove them yourself if needed. It's also a good idea to have gloves handy to protect your hands while you're lifting and applying fake eyelashes.\\\", 'First, use a toothpick or similar object on your lashes. If your eyelashes are too long and you prefer soft lashes, a toothpick is perfect.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}