{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac9ed53b-e276-5614-91be-b1eb6bae3c5f","task_key":"default--validation--13146","task_revision_id":"1","upstream_id":"13146","short_description":"HellaSwag validation 13146","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to write a wedding speech [title] Introduce yourself to the crowd. [step] Start by letting everyone in the bridal party and crowd know who you are. Tell them your name, role in the wedding and relation to the people getting married.\",\"endings\":\"[\\\"Not everyone will have met you, and they'll want to know how you're connected with the bride or groom and why you've been asked to give a speech. [substeps] The best man and maid of honor are usually each asked to make a short speech on behalf of the wedding party.\\\", 'Avoid personal comments like \\\" i don\\\\'t know you, \\\" or \\\" this is my first wedding, \\\" because they will likely have no idea of who you are. [title] Proofread all of your work thoroughly.', 'Next, use your stage name, if you\\\\'re traveling it should be a sophisticated, clear name, such as \\\" jake monroe \\\", you could also use \\\" bee _ cupcake \\\" or \\\" bass frog from blue cloud. \\\" [substeps] Get more specific if the party is getting loud and the bride and groom argue.', 'When you create your speech, you should then introduce yourself to the people you would be parting with. This will get them thinking about you and your preparation for the speech.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}