{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"758caeab-52c1-5451-90ab-3ef49cc143cb","task_key":"default--validation--27748","task_revision_id":"1","upstream_id":"27748","short_description":"HellaSwag validation 27748","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a haircut last for six months [title] Ask your stylist about the right cut for your hair. [step] Be open with your stylist: you want a cut that will last a good while. When you make that request, ask what would be most appropriate for your hair type.\",\"endings\":\"[\\\"If your stylist doesn't know, it won't necessarily go into effect until you take measurements. [substeps] Some suggestions for asking include: if you usually wear your hair down, let it grow out a bit.\\\", \\\"If your stylist cuts to your hair type, it's more likely to last than if they don't. [substeps] For instance, finer hair may work best with less layering.\\\", 'Many people have hair that appears very green, but this particular type of hair-coloring has not been shown to have any similar advantages. If your hair is undernourished and damaged, you could end up with hair with coarse, brittle undertones.', 'Just the answers you get will not make you the most successful stylist. [substeps] Look at the materials that your stylist is using to create your cut.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}