{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"967aa2d2-c15d-5bce-aa6e-7c19863746e2","task_key":"default--validation--21203","task_revision_id":"1","upstream_id":"21203","short_description":"HellaSwag validation 21203","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to be a vendor at a convention [title] Come prepared. [step] This means three things : [substeps] Have something attractive to draw people to your table. This could be a clever/shocking/attractive sign, a large picture, or big, bold text.\",\"endings\":\"[\\\"Simple and solid is your guide to attracting people as they arrive. Try not to draw them in with items new you've collected or items with extra or super attractions because they will always think they're the only one that's special.\\\", 'If anyone has heard of these things, get it in the local newspaper and ask around. Pre-packaged food should be elbow-deep in the mix.', \\\"Posters and tri-fold pieces of poster board are very common. This is actually the least important part of coming prepared-but it really does help! Know what you're going to say ahead of time.\\\", 'You could also have an interesting and well-detailed pic. Be sure to have it on your table and that you follow appropriate etiquette.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}