{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dec055ca-942f-53df-a7bd-25faeabe1ea8","task_key":"default--validation--34778","task_revision_id":"1","upstream_id":"34778","short_description":"HellaSwag validation 34778","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose an alternative to a flower girl [title] Invite other members of the family to join your wedding party. [step] Instead of having young flower girls, consider using older women in your family, such as your grandma or great aunt. They'll love to be included in a special way, and the sight of your grandma throwing flower petals into the air is to sure to bring a smile to everyone's face.\",\"endings\":\"['[substeps] Older women can even invite their significant other as well. Just be sure the couple makes time to discuss the details in advance.', \\\"[title] Let everyone know you're having a flower girl themed party. [step] In several ways, this will make your party more fun.\\\", '[title] Read opinions from other members of the family. [step] Reading magazines and articles on the flower girl instead of relying on your friends for opinions can help you understand what happened that night in the picture.', '[title] Give the role to a friend who volunteers. [step] One of your adult friends may find the idea of being a flower girl enchanting.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}