{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ea80a6f9-6992-558c-8081-aad1b3b0a73a","task_key":"default--validation--13487","task_revision_id":"1","upstream_id":"13487","short_description":"HellaSwag validation 13487","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose and wear cologne [title] Wear cologne with oriental or wood notes in the fall and winter. [step] You can choose to wear cologne with the seasons by avoiding light and floral scents in the cold months and instead wearing earthy colognes like sandalwood, cedar, and vetiver to complement the weather. [title] Choose a cologne with floral and fresh notes in the spring and summer.\",\"endings\":\"['[step] The musky scent of wood will pair better with flowers, oils, and gels. [title] Choose a cologne with a classic or vintage appearance.', '[step] As the weather gets warmer, wear cologne that enhances the season. Floral, fruity, green, water, and citrus scented colognes are great for the warmer months.', '[step] You can choose to wear your cologne with any time of the month and on any day of the month. Whether you wear it during spring or summer will depend on the season.', '[step] Off season perfumes have floral and fresh notes throughout a season. Choose a cologne with floral and fresh notes at every season to help preserve and cover its aroma.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}