{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1eb8bb25-2087-5932-a340-a769eb3c4249","task_key":"default--validation--42413","task_revision_id":"1","upstream_id":"42413","short_description":"HellaSwag validation 42413","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to become the centre of attention [title] Stand in a central position. [step] If you are at a party or another type of social setting, you'll receive more attention if you stand in a prominent place. [substeps] If you're in a social setting with friends or people you know, ask them to join you in the center of the room.\",\"endings\":\"['This will help you appear friendly and connected; new individuals will be drawn to you and your friend group. [title] Move locations every 15-20 minutes.', \\\"This will allow your friends to signal that you're ready for conversation. If your friends don't know where to begin, ask them to move back to one place next to you.\\\", 'If not, sit on one end of the table and disappear to the other to avoid wandering into a space (this will increase the chances of moving away from the group without getting noticed). Ask friends about what they think of your new center of interest.', \\\"No matter if they don't want to join you, this approach will help keep people engaged. If you can't be seated at any one spot, sit in a more cramped position.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}