{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"318ff5f7-83cb-52b9-a92c-dfeb8f55e13b","task_key":"default--validation--19711","task_revision_id":"1","upstream_id":"19711","short_description":"HellaSwag validation 19711","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear over the knee boots [title] Put on an oversized sweater and scarf. [step] Over-the-knee boots can be difficult to wear during cold weather because they can't be worn with bulky pants. Instead of opting for only thin clothing, wear bulky clothing over skinny pants, leggings, or tights.\",\"endings\":\"['Wear a thick, oversized sweater or sweater dress. Pair a large scarf with the sweater and your basic outfit is complete.', '[substeps] If the weather is warm enough, add pants, gloves, a sweater, or jacket. Pair a brightly colored sweater with a plain sweater to create a balanced look.', '[substeps] Layer one sweater in a neutral color like brown, with an oversized crop top. Take a bright scarf and tie it around your neck.', \\\"[substeps] Try wearing a long-sleeved white t-shirt over skinny jeans if you're going for a relaxed presentation. Pair your stretchy sweater with a pair of snug black leggings.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}