{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"744e0c44-7f1f-578b-b92f-538347c13850","task_key":"default--validation--19353","task_revision_id":"1","upstream_id":"19353","short_description":"HellaSwag validation 19353","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to add friends on snapchat [title] Find out your friend's username. [step] The main way people add each other in snapchat is by trading usernames. This allows you to add people that aren't in your phone's contact list.\",\"endings\":\"[\\\"Each time a person will be added to your phone's contacts list, you save your current chatting list. [substeps] Snapchat has a feature to fit most smartphones-whatsapp, safari, or similar.\\\", 'Since snapchat is a long-distance connection between people, you can use it to your advantage and get friends with similar names. [substeps] Snapchat is often used on facebook and other social media platforms.', \\\"There is no way to add someone to snapchat using an email address or real name. You'll need to ask the person what their snapchat username is.\\\", \\\"Many people swap their usernames with a different user in order to easily get more followers. This doesn't give you enough available space to add two people.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}