{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b644a1d-c222-5a8c-b218-b0aee7d2541a","task_key":"default--validation--22076","task_revision_id":"1","upstream_id":"22076","short_description":"HellaSwag validation 22076","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a denim dress [title] Consider the wash of the dress. [step] Denim comes in a variety of colors, but most denim dresses are usually some shade of blue. However, you still have options when it comes to the depth of the blue.\",\"endings\":\"['Some denim washes work best for more polished looks, while others are ideal for a relaxed outfit. [substeps] Dark wash denim tends to have a more dressed up look, so they work well for work or an evening out.', \\\"Depending on your dress's style, some of the colors will show up darker, and others won't. [substeps] In general, the darker the shade of the denim, the darker the dress.\\\", '[substeps] Jeans tend to look better when washed and ironed on both sides. Likewise, jeans can also be washed and ironed on both sides of the dress, but with less work involved.', '[substeps] Look for dresses made of the same material or a lighter blue. Choose washes made of the same material, usually denim colors such as fuchsia, navy, and tan.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}