{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81897bc6-7d54-5a6f-9cd3-aed246a1e9b3","task_key":"default--validation--27778","task_revision_id":"1","upstream_id":"27778","short_description":"HellaSwag validation 27778","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to announce the bridal party at a reception [title] Give the mc a written list of names. [step] Your wedding party is a big part of your special day. When you introduce them, you want to make sure that they get the recognition that they deserve.\",\"endings\":\"[\\\"Count up to fifty names at the reception and give each one two days' notice. [substeps] Making a list of names is a great way to tell if your reception is going well.\\\", 'Make sure the dj or mc has a written list of the names of the entire bridal party. [substeps] If necessary, include a pronunciation guide for the announcer.', 'List their contact information as soon as possible, because this gives the mc time to do the actual announcement, or else you could miss out. [title] Note the formality of the wedding.', 'Write down the names of everyone involved, the names of the bride, and the groom. Have them read it to the camera for the party and make sure all the names are correct.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}