{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa3b1e5c-5f33-5e7b-9532-d32bab4b2479","task_key":"default--validation--29382","task_revision_id":"1","upstream_id":"29382","short_description":"HellaSwag validation 29382","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a leather jacket [title] Wear a bomber jacket for warmth. [step] Bomber leather jackets are waist-length with a soft lining. They tend to be the warmest variety of leather jacket.\",\"endings\":\"['The fleece layer of the jacket covers most of the alghts, allowing you to warm the jacket in the winter and colder in the summer. [substeps] Single-breasted bomber jackets are also the most common style in many colder climates, so feel free to experiment with your style.', \\\"That being said, they are even best when you're not playing the sport. These leather jackets are generally better to wear when you're playing hockey or soccer or definitely when you're exercising.\\\", 'Before going outside, wear a bomber jacket for extra warmth. For colder days in the fall, wear a bomber jacket for companionship and warmth.', \\\"If you're wearing a leather jacket for a winter look, go for a bomber jacket. [substeps] Bomber jackets are also more casual, so they can work great for a casual winter look.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}