{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99ea2dd4-3a35-5a90-a96e-9fe2888cb288","task_key":"default--validation--36956","task_revision_id":"1","upstream_id":"36956","short_description":"HellaSwag validation 36956","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look rich without being rich (for guys ) [title] Dress smart. [step] Wear shorts only to the beach or when shopping. Instead of always wearing t-shirts, try wearing men's plaid shirts over jeans or khakis.\",\"endings\":\"[\\\"If you are using men's stripes, then don't sweat it. Just wear a yellow button down shirt with women's stripes that's nice on you.\\\", \\\"Nothing too fancy though, because it wouldn't be appropriate. If you don't have the money, find old jeans with fashionable creases and add them to your wardrobe.\\\", \\\"It's a look that's the same everywhere, and you'll look wonderful wearing it. Think people's looks, not your.\\\", 'In more formal circumstances, wear a jacket over your shirt. Dress up in a dress suit once in a while.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}