{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d738276-4f52-5af0-95c0-6d665c377dee","task_key":"default--validation--538","task_revision_id":"1","upstream_id":"538","short_description":"HellaSwag validation 538","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a dress in winter [title] Try tights and leggings. [step] Opaque tights can be worn with short, mid-length, and long dresses to keep your legs warm. Instead of tights, opt for leggings for those extra cold winter days and nights.\",\"endings\":\"['Have a pair of tights for those freezing cold days. You can also wear or opt for leggings in a neutral color for a warm, chic look.', 'Keep in mind that nude tights and leggings (black, grey, and cream) go well with any dress. [substeps] Keep it fun by wearing colored or patterned tights and leggings; for example, match a black dress with red or purple tights.', \\\"[substeps] Cold weather cuts down on clothing and can get stinky hot as well if you're cold. Allow yourself to sweat for at least three hours in extreme temperatures.\\\", 'Also, look for tights made of a breathable material like cotton as they can help keep you warm. [substeps] You can find tights at almost any clothing retailer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}