{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8352a14c-2720-56ac-9f3c-96a1d15b1694","task_key":"default--validation--42397","task_revision_id":"1","upstream_id":"42397","short_description":"HellaSwag validation 42397","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to have fun at younger children's parties [title] Hunt for a gift to take to the party. [step] Some great ideas are toys, of course, but you can also be a bit more creative. Put together a \\\" gift pack \\\" filled with coloring books, crayons, and small games.\",\"endings\":\"[\\\"Make sure the toy is what you want it to be, not what you're just shopping for. You don't want to come out looking miserable.\\\", \\\"Some of the simplest things can bring entertainment to a child. Also, if you're at a complete loss for what to give the child, you can get a card and put ten bucks in it so they can buy something that they know they will enjoy.\\\", 'Anywhere from a few hundred to hundreds can make a great gift, from small dolls to miniature cars. Hide-and-seek is also an important theme.', 'Make a guilty gift at the party, maybe a cute picture of your best friend or the prettiest toy one of your friends bought. After all, this is your birthday celebration, too.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}