{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2524717-9107-5353-8ec5-429c692c193d","task_key":"default--validation--4622","task_revision_id":"1","upstream_id":"4622","short_description":"HellaSwag validation 4622","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach kids to wear glasses [title] Explain the need for wearing glasses. [step] Tell them that if they can't see or read something because it is too far away, it's because they don't have their glasses on. Reassure them that the glasses are a normal thing, and no one worth his/her time will mind.\",\"endings\":\"['Even when the kids ask why they can\\\\'t see their glasses, explain that learning also makes them smarter and helps them to visualize objects better. [substeps] You could say, \\\" nobody can see exactly what you\\\\'re doing without glasses on.', 'Explain, in simple terms, depending on the age of your child, how they work. [title] Let them think of it simply as a \\\" full-time accessory \\\".', \\\"Explaining that glasses are a part of education looks good on older children at home. [substeps] If your child isn't reading at home, explain to them why this part of their education will look good on older children, if they have their glasses on.\\\", 'Explain to them that you are a parent trying to help them learn to read well. [title] Explain to them that using glasses makes them laugh.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}