{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e02c9e0d-1add-58ef-b0ab-c7cce69b9e24","task_key":"default--validation--7021","task_revision_id":"1","upstream_id":"7021","short_description":"HellaSwag validation 7021","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to find the perfect pair of sneakers [title] Consider your current wardrobe. [step] The perfect pair of sneakers will look like they were made just to be part of whatever outfit they're worn with. However, since you probably can't have a closet full of sneakers, consider the clothes you have beforehand, so that you can make sure your sneakers match you clothes as much as possible.\",\"endings\":\"[\\\"[title] Ask a friend to lend you some good advice. [step] You'll be saving money just by borrowing a pair of converse/vans shoes from someone you know.\\\", 'The best rule of thumb is to make sure that the shoes you have are not too short or too baggy. This will help your lines match better with your clothes and make you look better.', '[substeps] Try wearing a pair of black cropped jeans. A black long-sleeved button-down (with a sweater instead of a cardigan) will look great paired with a white button-down shirt.', \\\"Once you have an idea of what colors you're looking for, proceed. If you plan on buying tons of new clothes in the near future, skip this step.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}