{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"127f570b-4d51-514c-a8cd-10a20aa5da00","task_key":"default--validation--13355","task_revision_id":"1","upstream_id":"13355","short_description":"HellaSwag validation 13355","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to wean your baby from your bed [title] Talk to your baby about the transition. [step] Read him books about babies and children who sleep through the night. Talk about how happy it makes mommy and daddy when he sleeps until morning.\",\"endings\":\"['Research shows that you can gradually wean your baby from the diapers and your mattress when you wake up. [title] Play \\\" akies \\\" with your baby.', 'Make a point to emphasize that daytime is for playing and having fun, while nighttime is for sleeping and relaxing. Remember to reinforce every little step he makes with praise.', 'Think of a fun bedtime routine for him to begin to care for. You can also cook healthy meals he enjoys.', 'Talk about how the arrival of twins can make his daily life even more special. [substeps] Read \\\" mommy \\\" by joan hamry when he wakes up and \\\" daddy \\\" by lara ross when she goes to school for the first time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}