{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"942fd17d-f09c-5491-9f93-57f467b42d37","task_key":"default--validation--13280","task_revision_id":"1","upstream_id":"13280","short_description":"HellaSwag validation 13280","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to help your child sleep on a plane [title] Book red eye flights and night flights. [step] Night flights are considered the last flights that leave the airport for the day; red eye flights are the flights that leave at night and do not land at their destination until the next morning. These types of flights are recommended because your child will already be sleepy and less stimulated because it is nighttime.\",\"endings\":\"[\\\"Landing times are also dependent on your budget. Ask your child's airline what airline is the best for your child.\\\", '[substeps] Pick out a window seat for your child. Your child will feel more protected because she gets to sit between you and a wall.', \\\"If this is not a concern, you can book flights and types of flights based on your kid's needs. [substeps] Prioritize your priorities in connecting to your kids.\\\", 'The flight will arrive at the destination between 10 a.m. and 9 p.m.; however, this is not a best time to book.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}