{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c15ec18f-a503-53f6-8bf4-ef879d84aebe","task_key":"default--validation--37137","task_revision_id":"1","upstream_id":"37137","short_description":"HellaSwag validation 37137","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear ankle boots with jeans [title] Opt for bootcut jeans. [step] The term \\\" bootcut \\\" isn't just for show. These jeans are fitted around the hips but flare out slightly below the knee, making them the ideal cut to wear over ankle boots and other boot styles.\",\"endings\":\"['[substeps] Boots come in a multitude of patterns, such as military chukkas or harvest shanks. Hiking boots are often paired with ankle boots since toe boots are more fashionable.', 'Bootcut jeans can be worn in both casual and everyday wear, just as long as the bottom hem remains on all day long. [title] Buy your ankle boots from a retailer.', 'Bootcut jeans are more casual and, on the contrary, easy to wear. [title] Go for cinch-in cargo boots.', '[substeps] Be careful when pairing ankle boots with jean styles that are cut wide down the length of the leg or those that flare out dramatically from the knee down. If you have shorter legs, these styles can make your legs appear even shorter when paired with ankle boots.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}