{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2473615-e325-5dd0-9b74-14f3980e4cf0","task_key":"default--validation--43633","task_revision_id":"1","upstream_id":"43633","short_description":"HellaSwag validation 43633","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to deal with enemies [title] If you don't know why they hate you, ask them calmly and nicely. [step] But don't approach your hater, if you know what they are capable of something drastic or something that can leave you hurt. Or it could leave you in embarrassment.\",\"endings\":\"['[title] Make a joke to taunt them and \\\" make fun of me, \\\" and then walk away, and chances are soon you\\\\'ll do it again. [step] [title] Be sweet, and think of something worth looking forward to when you talk to them.', \\\"[title] Don't beg and refuse to be approached. [step] This will give your enemy more power over you in the long run.\\\", \\\"If they are nasty to you or start swearing or being violent, walk away and don't bother trying to talk to them again until they start being civil. Some people can be given second chances to realize that you aren't really a bad person and they have nothing to hate you for.\\\", \\\"[title] When talking to your hater, calmly but firmly tell them that your relationship with them isn't over and that they won't see you again. [step] Also make it clear that you respect them from now on.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}