{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc8cce7c-d89b-5789-844b-3ebddce23d11","task_key":"default--validation--10757","task_revision_id":"1","upstream_id":"10757","short_description":"HellaSwag validation 10757","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to find a life partner [title] Put yourself out there. [step] The more time you spend dating and socializing, the more likely you will find someone you like. Put yourself out there by attending social events organized by your friends, by taking classes and chatting with classmates, and by signing up for dating sites, apps, and services.\",\"endings\":\"[\\\"[substeps] Constantly keep the number of dates you've already had in mind handy, using a number of these to boost your social sense. Though you'll find somebody attractive, you'll definitely land someone who doesn't have the same hang of social interactions as you.\\\", 'Be adventurous and open minded: try speed dating, for instance. [substeps] The most common way to meet a future partner is through mutual friends.', 'The more you find a partner, the more time you will have to devote to them. [substeps] This can be any musician, movie star, or comedian.', '[substeps] For example, invite your friends to sleepovers, go to a movie theater with a friend, or take up reading in your own class. Take a class in your chosen subjects so you feel like you have a real connection with the person.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}