{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e0980e8-1257-5755-bbc9-aaac9b3ae281","task_key":"default--validation--24666","task_revision_id":"1","upstream_id":"24666","short_description":"HellaSwag validation 24666","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress fashionably [title] Organize your wardrobe. [step] Get out all your clothes and decide which ones you want and don't want. Donate, sell, and a top tip you could host your own boot sale, with anything that you haven't worn in a year, doesn't fit or isn't your style.\",\"endings\":\"['Pull out from the possibilities what would suit you and get a huge crowd. At least one large outfit is a reasonable estimate.', 'Also, pile your clothes up--keep them in two stacked piles so you can pull them out if they fall out. [title] Avoid clothes that are too trendy.', '[substeps] If you haven\\\\'t worn it in a year, you won\\\\'t miss it. Thinking, \\\" i could need this one day! \\\" will leave you thinking you have nothing to wear for most.', \\\"Many of the people who come into this store regularly get dress shirts, trousers, jeans, and other outfits. You can find a directory here to list what they sell, or you might have to go to your parents' house to shop for alterations in wardrobe.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}