{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"005770dd-4dcd-59c5-a1b4-3a48222135cd","task_key":"default--validation--42169","task_revision_id":"1","upstream_id":"42169","short_description":"HellaSwag validation 42169","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to inspire creativity in your kids [title] Be a role model. [step] Be open-minded in your thinking and find lots of solutions to many problems. Show your kids that you are flexible and willing to try new things.\",\"endings\":\"['When faced with difficulty, show your kids that you can approach the problem in different ways and still be okay. [substeps] If your kids ask you a question, be creative in how you answer it.', '[substeps] Consider starting your own art class for your kids. This improves their understanding of what art means.', \\\"Tell them there are plenty of excellent ways to reinforce what's already in front of them. Young kids can ask many questions of themselves, and you can provide the biggest answer.\\\", \\\"They'll be more likely to succeed if they feel they can see this side of yourself. Be flexible in your thoughts, thinking on your toes, and feeling comfortable in your own surroundings.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}