{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03d611b0-5791-5482-8d94-a8b2e475f1b7","task_key":"default--validation--44914","task_revision_id":"1","upstream_id":"44914","short_description":"HellaSwag validation 44914","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look scary in a rough neighbourhood [title] Wear clothes to fit in. [step] If you are in a rough neighborhood, most people there are probably not wearing expensive suits, lots of bright jewelry, and shiny shoes. You need to wear clothes that make you fit in, not stand out in a way that makes you a target.\",\"endings\":\"[\\\"[substeps] Dress in gendered clothing, whether it's a polo shirt with biker pants or a comfortable leather jacket over a skinny jean skirt. This will give off a strange vibe that isn't appealing to most people.\\\", 'For example, a shirt and a sweater, an funky skirt, and a dress would make a good way to look scared. If you are out playing or sticking up for people who are into drugs or alcohol, wear any clothes that clash with you.', \\\"Dresses and leggings should be comfortable and can go with almost anything. Make sure they aren't too tight or too short to walk in.\\\", \\\"Choose simple clothes such as jeans, sweat pants, plain t-shirts, sweatshirts, and coats that don't appear new. Consider wearing clothes that are kind of beat up, maybe even dirty.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}