{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"acd3d493-302a-5e37-9ea8-0885986f9956","task_key":"default--validation--4292","task_revision_id":"1","upstream_id":"4292","short_description":"HellaSwag validation 4292","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to travel by plane with an infant or toddler [title] Use an approved restraint if possible. [step] Lap travel, while legal for children under 2 years of age, is much less safe than traveling in an approved seat. The faa recommends that all children use a restraint on board from birth to 40 pounds.\",\"endings\":\"['[title] Use a seatbelt if you are traveling over distances of less than 4 1/2 meters. [step] Less than 4 , 000 feet (8 , 260 m) can cause significant harm to an infant or toddler.', 'Although each airline has additional regulations regarding proper safety precautions, these regulations can be handled by the faa. [substeps] The faa requires that all passengers be licensed, tested, and insured, from the age of 10 to 17, for at least a year.', 'For all states, even though safety restrictions may be in place, you should still consult the faa before making your trip. The americans with disabilities act of 1967 prohibits children under the age of 14 from dangerous activity.', 'It also comes in handy because you can attach toys or pictures to the top, a mirror to the side, and a pillow to the back for comfort while sleeping. [substeps] However, in europe, australia, asia, the use of a lap belt onboard is normal and approved by their safety regulations.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}