{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fde60da-8360-537f-ac8a-e8b46ffc1005","task_key":"default--validation--31027","task_revision_id":"1","upstream_id":"31027","short_description":"HellaSwag validation 31027","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look and feel stylish (for men ) [title] Have a reference point. [step] Think about your personal style icons: people you consider to be the height of style. Whether it's kanye west, ralph lauren, james dean, or walt whitman, try and wonder what they're trying to accomplish with their style.\",\"endings\":\"[\\\"Don't just judge people based on age, gender, or how they stand out to others. Focus on physical features that you can relate to, such as their eyes, hair, beard, etc.\\\", \\\"Think carefully about what they're trying to achieve. [substeps] For example, who would go for a low-contrast look with wearing jeans and a large t-shirt just to camouflage their skin? Think about it more.\\\", 'If it is truly \\\" the perfection of a point of view, \\\" think what your icon\\\\'s point of view might\\\\'ve been, and think about what your point of view might be. Whatever it is, let it guide you when you shop and dress.', 'Some people are more dramatic in appearance and fashion, and others have a more laid back image or appearance. [substeps] Consider using your-mark measurements from the chart below for your plus groups.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}