{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"077aeed2-bd18-5b07-a7b3-f7f190c8c4b0","task_key":"default--validation--450","task_revision_id":"1","upstream_id":"450","short_description":"HellaSwag validation 450","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to develop social skills in children [title] Respond to your baby's needs. [step] Babies are unable to tell you what they want directly. This means you will need to pay attention to the actions and non-verbal cues that your baby gives.\",\"endings\":\"['Without being present, your baby will often wait for you to respond. Most babies start to follow instructions and reactions that are expected to occur.', 'When you have your baby in your lap, look them in the eyes and talk to them very softly. If they are not aware, their attention may wander and they may struggle to respond to your words.', 'They will more than likely have questions they want asked. Raising a child and providing emotional and physical support are both natural roles for your child, and protecting him or her from problems could involve a lot of stress.', \\\"When you know what your baby needs, give it to them. If that doesn't work then you may have misinterpreted their needs, and you should try something else.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}