{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54609eef-fcf5-58c6-bf83-8ffc257737a0","task_key":"default--validation--41698","task_revision_id":"1","upstream_id":"41698","short_description":"HellaSwag validation 41698","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to tell someone you still love them [title] Make sure that you mean it. [step] If you're going to tell someone that you still love them, you should be certain that you do, indeed, still love them. Try to gain a clear perspective on the situation.\",\"endings\":\"['Clarify why you are telling this person that you still love them, and try to imagine how he or she will react. Ask yourself what you hope to gain from saying this.', \\\"Are your feelings so strong that you don't think about or act upon them fully? [title] Walk away from them. [step] If you know you do still love them, that doesn't mean you're crazy.\\\", \\\"Remember how others have handled this situation in the past, and how you've done it differently. Figure out exactly what you mean by love to you, and stick to it.\\\", \\\"[substeps] You don't want to come across as clingy when you say it out loud. Instead, try to convey that you just want to be friends.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}