{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6812e562-af62-5845-b210-6c8c86e69e32","task_key":"default--validation--46983","task_revision_id":"1","upstream_id":"46983","short_description":"HellaSwag validation 46983","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan your child's first playdate [title] Teach your child how to be a good host. [step] Your child will likely not understand the concepts of good hosting unless you teach them. Explain that good hosting helps guests feel welcome.\",\"endings\":\"[\\\"Over time, this will be a great opportunity for your child to network and be comfortable living together. [substeps] Your child's goals and understanding of a house could help them put their minds at ease when the two of you go on your first date.\\\", 'Playdates are more fun when everyone feels like they are wanted there. [substeps] Good hosting begins with greeting the guest at the door.', 'If your child has never been to a play, talk to your child about letting your child be active in a soccer game at a local playground. Encourage your child to be able to help others play it well.', \\\"[substeps] Explain to your child their obligations before you introduce a new guest. This may involve teaching your child the purpose of help cooking or taking place under your house's roof.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}