{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df53eff4-e872-5deb-ba64-865d6d65871a","task_key":"default--validation--44251","task_revision_id":"1","upstream_id":"44251","short_description":"HellaSwag validation 44251","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear leggings to work [title] Choose black leggings. [step] If you're a fan of leggings, you're aware that they come in every color under the sun. While a pastel pink or a cool patterned pair are great for a workout class, don't try to dress those up for work.\",\"endings\":\"[\\\"Stick to grays and browns, so the pair you choose makes up for a drab look. [substeps] You can even go with both patterned and neutral leggings if you're adding variety.\\\", '[title] Select black leggings if you want to be better at show-dressing. [step] Studies show that body-builders tend to favor black leggings as a part of their choice of workout attire.', \\\"If you want to pull off leggings at work, they need to be black. Black leggings look much more sleek and professional than any other color, and you're less likely to call attention to the fact that they are, in fact, leisure clothes.\\\", \\\"They're too neat to work and will likely fail to convey their workout vibe. Opt for knee-length leggings in black or an olive color that's around knee-length.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}