{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c02c38ea-4c12-5c24-96df-4598571df0c9","task_key":"default--validation--25337","task_revision_id":"1","upstream_id":"25337","short_description":"HellaSwag validation 25337","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to keep up with fashion trends [title] Watch the runway. [step] Fashion brands and designers have shows twice a year at designated \\\" fashion weeks \\\" in various cities around the world. While it is difficult to gain access to these events, there are many resources available online that recap the shows with photos, videos, and written reviews.\",\"endings\":\"['Pay special attention to points like : [substeps] Where and when you can get around. How you stores often, how many places you go, where the clothing looks great, and how many different shoes you can buy, etc.', 'Two brands offer unique styles for one fashion leg, the waist, in addition to addressing many of the other social events of which authors claim they work. Seeing them on television, featured style magazines, , etc.', \\\"When you are reading or viewing official fashion shows online, get an idea of what exactly you are going to enjoy doing each week for a day. [substeps] Save your vhs reports for study days, as they'll introduce you to more exciting clothes.\\\", '[substeps] Style.com and nowfashion.com are two reliable places to see recent video footage and photographs from runway shows. You can also watch archived shows from past years to get an understanding of how trends have changed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}