{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98161d7d-da95-50c9-9993-23ed699379ae","task_key":"default--validation--7297","task_revision_id":"1","upstream_id":"7297","short_description":"HellaSwag validation 7297","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to protect your infant from flu [title] Get vaccinated. [step] As soon as the seasonal flu vaccine is made available, you and other caregivers should visit your doctor and get vaccinated. By being vaccinated, you'll protect yourself from being infected.\",\"endings\":\"['As a result, your chance of infecting your infant will be much lower. Without a flu vaccine, you and your family will be much more vulnerable.', '[substeps] First, you will need to get vaccinated when you are 15 years old or older. Remember that vaccines do not prevent the flu from occurring in children.', \\\"All viruses can lead to flu, but there are also vaccinations that prevent a potential recurrence. They're not as important as any other vaccines, though.\\\", \\\"[substeps] Vaccinations can last for several months. Contact your doctor immediately if you've had the flu in the past, have just become pregnant, or if you have recently given blood.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}