{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9885e414-023c-5eb5-9bf0-418d844a2b9d","task_key":"default--validation--34107","task_revision_id":"1","upstream_id":"34107","short_description":"HellaSwag validation 34107","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress thrash [title] Obtain band t-shirts. [step] Black is generally the color most associated with metal fashion, thrash is no exception. A local metal shop would be your best bet.\",\"endings\":\"[\\\"You don't want a lot of people dressing sporty, and if you live somewhere with heavy traffic, find shirts and tops that sell bands. [substeps] For instance, if you find band t-shirts made from ripped vinyl, go for those.\\\", \\\"If you don't have your own band t-shirt, you can go for a great-arm tee. Dress out the idea with a band t-shirt that looks black, with or without artwork to accessorize the band's logo.\\\", \\\"Band t-shirts are often bought at dmv stores with many different styles featuring the band. Some sleeves are extra, but you can't go wrong with tops.\\\", \\\"If you don't know any, try the internet, more unique shirts can be found there. Aim for thrash shirts, death and black would also suffice if you listen to these genres.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}