{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89e4fa43-f85e-5b13-b880-8c43a2ced8be","task_key":"default--validation--38054","task_revision_id":"1","upstream_id":"38054","short_description":"HellaSwag validation 38054","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear harem pants [title] Select the fabric style carefully. [step] Some fabrics will drape more elegantly than others. Try opting for silks, cotton jersey, and other delicate fabrics that let the pants drape in all the right spots.\",\"endings\":\"['Choose a rich, airy fabric that will also blend in with the other fabrics in the range. [title] Select a pleated skirt for a laid back style.', 'Silks are versatile fabrics that change fairly easily or are easily pulled. But ones that drape more generously on the legs and feet, especially the waist, bring out a feminine look.', 'If it seems like the fabric may easily wrinkle, search for a different pair. These pants already have a casual vibe because of their fit, so it is important to find fabric that will stay wrinkle-free.', \\\"[substeps] Harem pants can be changed around, but you don't want to be wearing harem pants in the middle of winter. [title] Opt for a pair of thicker harem pants.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}