{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8907da81-fa38-56ae-a3c6-d3751066687e","task_key":"default--validation--21937","task_revision_id":"1","upstream_id":"21937","short_description":"HellaSwag validation 21937","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear yoga pants to work [title] Invest in high-quality yoga pants. [step] If you're wearing yoga pants to work, you want them to be durable while also providing comfort. Try to find yoga pants made of a thicker material, and try them on before purchasing them to ensure they aren't see-through.\",\"endings\":\"['High-quality yoga pants will likely be designed for yoga or other types of work. [title] Stick to a thin, stretchy, short-waist.', 'Sheer yoga pants are definitely something to avoid in the workplace. [substeps] Yoga leggings offer the most outfit options and tend to look the best in a professional work environment.', '[substeps] Make sure the pants are specifically made for yoga, preferably ones with natural fibers, such as cotton or silk. When in doubt, look for yoga pants in solid colors.', 'Yoga pants come in a variety of shapes and sizes, and none of them fit under normal circumstances. See the rough guide section for more specific instructions.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}