{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fe59ec9-0062-5dbe-816f-1a150fb1a7b5","task_key":"default--validation--26933","task_revision_id":"1","upstream_id":"26933","short_description":"HellaSwag validation 26933","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to have a great sleepover birthday party [title] Make the invitations. [step] Make the invitations in such a way that it reflects your personality. [substeps] You can use microsoft word to make them, or write them by hand.\",\"endings\":\"['Color them with decorative writing if you have time to do it yourself. Make sure the images are-pictures-of you, not simply an anonymous character.', '[title] Introduce your friends and add a special candle or a birthday themed object, such as a cute picture of candy. [title] Decorate the rooms with fun things to include (or a video game to play), colorful balloons, silly people writing random things to remember, and maybe some chocolate chip boosters.', \\\"It doesn't really matter, as long as the guests know all the information. On the invitations, include who the party is for, what day it is, what time it starts and ends, where your house is, and your phone number (rsvp).\\\", \\\"Sending your invitations can be fun, but they often don't do the trick as well. Invitation-makers were generally skilled at assembling their invitations, which they say is best for you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}