{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ae0eb8b-0eb0-52df-86e0-20c46f9db43b","task_key":"default--validation--48946","task_revision_id":"1","upstream_id":"48946","short_description":"HellaSwag validation 48946","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to pack a diaper bag for a plane trip [title] Pack a few of your child's favorite toys. [step] Bring age-appropriate items like chew toys, cars, doll, or stuffed animal. Try not to bring things with many pieces, so there's less to keep track of.\",\"endings\":\"['[title] Pack a first aid kit so you are prepared for any injury. [step] Make sure your kit includes bandages, gauze, antibacterial ointment, cotton swabs, tissues, alcohol wipes, thermometer, and hand sanitizer.', \\\"Be sure that the toys are first-line or that your child has enough of each type. [substeps] Buying early is also recommended because it saves you a lot of time and effort on buying too many toys than you're likely to have at some point.\\\", 'Keep the variety of toys you pack in mind, either in a suitcase or buying. A suitcase will save a lot of unneeded space on the plane and enough room for the litter box, shampoo, and first aid kit.', \\\"Prioritize what you'll need each time, so that you're quick about bringing everything your child needs the most. [substeps] These things are usually things that you don't want your child to forget in case an emergency arises.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}