{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7bdc9a0a-3490-5ede-aa41-d3df1a8bc709","task_key":"default--validation--2578","task_revision_id":"1","upstream_id":"2578","short_description":"HellaSwag validation 2578","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a french pedicure [title] Remove any existing nail polish. [step] Soak a cotton ball with nail polish remover and gently rub the cotton ball on the surface of the nail. Keep rubbing until you see that the polish is completely removed.\",\"endings\":\"['You may need to use a bit of pressure to remove the polish entirely. [title] Soak your feet in hot water.', \\\"Alternatively, you can wash your hands with soap and water immediately after. You'll want to avoid accidentally rubbing the polish off.\\\", 'Then rinse the area with warm water. [title] Use nail polish remover to remove any sticker or adhesive that was on the base of your french pedicure.', '[title] Apply cuticle cream onto the polish. [step] Grip the cotton ball in one hand while gently rubbing the liquid part of the cotton ball into the nail and remaining polish to create a smooth shine.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}