{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83e645a8-740a-51c3-9bf9-916a0ed3ea9d","task_key":"default--validation--47210","task_revision_id":"1","upstream_id":"47210","short_description":"HellaSwag validation 47210","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to decorate a mailbox for the holidays [title] Choose a theme for your mailbox decoration. [step] Your mailbox decorations should correspond with the holiday you are celebrating. [substeps] Complement the decorations you already have in your home or around your yard.\",\"endings\":\"['Your preferred decor could be plastic tree decorations or intricate christmas decorations that your family enjoys. [title] Pick a theme for a mailbox if your hometown has few decorations.', 'Your christmas decorations must be more inviting to the holidays you are celebrating. Think about inviting local snow angels, owl owls, or a drift owl to your mailbox as an everyday decoration as well.', 'You could go for seasonal decorations that you have in your neighborhood, or you could stay minimalist with winter decorations that you have in your home. Themes that you are familiar with can help you then look for decorative decorations that will match the holiday in order to look festive.', 'For example, if you have a snowman theme, hang a snowman tree ornament from your mailbox. If you decorate your windows with easter egg decals, add them to your mailbox as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}