{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cb357da-628e-5aae-a4b8-a6f74bd9d7f4","task_key":"default--validation--4608","task_revision_id":"1","upstream_id":"4608","short_description":"HellaSwag validation 4608","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to care for a premature baby [title] Speak to the professionals about the problems that your baby is facing. [step] It can be very overwhelming and difficult for parents to understand exactly what their baby is at risk of. The staff is aware of this and generally will explain things to you in a way that you will understand if you continue to have questions and concerns.\",\"endings\":\"['[substeps] If you do not have professional advice, then educate your parents and work with an pediatrician to get your baby diagnosed. They should be able to provide you with some solid advice.', '[substeps] Fill out all of the paperwork prior to or after your appointment. Request copies of the drug and alcohol tests or any other testing the office has before and after your appointment.', '[substeps] Call the health and emergency services, drug and alcohol abuse hotline. If you feel wary about giving your baby anything, you can go a notch deeper and ask during a meeting for more information.', '[title] Ask as many questions as you need to in order to completely understand the care that is being provided to your baby. [step] Strange machines, special lighting, monitors and tubes attached to your baby can be frightening.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}