{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"decac423-7eed-5496-9300-0fe9f2bc371b","task_key":"default--validation--13816","task_revision_id":"1","upstream_id":"13816","short_description":"HellaSwag validation 13816","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be close with your teen child [title] Always keep your promises. [step] When you promise something, for example, to buy them something, or to take them some place make sure you always keep your word. [substeps] Never make empty promises.\",\"endings\":\"[\\\"Even a promise to do your teen's chores or to go to a club with them could end up being meaningless. Always promise to make the effort, even if you aren't serious about it.\\\", 'Avoid doing people favors that you know they will never do again. If you have to promise something, say it respectfully but never avoid it.', 'If you say you are going to do something, do it. If you have a genuine reason breaking a promise, be sure to explain it to your teen-aged child.', \\\"If your kid is planning on going to school, write down what you promise them and stick to it. Say this in the afternoon or when you're ready to pick your child up for school.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}