{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3da1d942-8356-5322-b195-5a853e3f0e77","task_key":"default--validation--9123","task_revision_id":"1","upstream_id":"9123","short_description":"HellaSwag validation 9123","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to snapchat a girl [title] Add her on snapchat. [step] Open snapchat and tap the ghost icon at the screen's top. In the resulting menu, select \\\" add friends.\",\"endings\":\"['\\\" tap the \\\" create new group \\\" button (start logging in one of the following ways). This will create a new group for you.', '\\\" below that is a list of friends who have previously added the person or you. Scroll down the list to tap a friend.', '\\\" after that, you\\\\'ll be able to add her with her username, through your phone\\\\'s contacts (though you\\\\'ll need to already have her number), or with a special snapchat code. [substeps] If you don\\\\'t have her username, number, or snapchat code, you can always add her by finding her in the friends\\\\' list of a mutual friend.', '\\\" a small window will pop up telling you if your selected people have successfully added friends. You can add plus one of her friends on facebook, twitter, or other other social networking platforms.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}