{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56caafc0-e4a8-5e4b-b3aa-45c0a4bb36fb","task_key":"default--validation--35379","task_revision_id":"1","upstream_id":"35379","short_description":"HellaSwag validation 35379","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to style leopard shoes [title] Match the style of shoe to the occasion. [step] Leopard flats are versatile and suitable to just about any occasion, but leopard print heels may be considered too flashy for most work environments. As such, it would be best to save those for a date or special night out, and stick with a pair of flats instead.\",\"endings\":\"['[title] Treat leopard print as a neutral color when composing your outfit. [step] This means that you can pair it with just about any color.', \\\"[title] Select high heels if you're starting out with a pair of leopard print stilettos. [step] Low heels increase the likelihood of your skin rubbing against heels, which will make them skinnier and therefore create more perspiration.\\\", 'Women, on the other hand, can wear flats to whatever event you would like. [title] Try a basic style of leopard for a touch of cushioning.', '[substeps] Leopard print flats can be knee-length, or longer. Just choose a contrasting colour, shape, or pattern that matches your dress code.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}