{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"caa51334-92b7-5c1a-ba9d-063cbfbf521b","task_key":"default--validation--5294","task_revision_id":"1","upstream_id":"5294","short_description":"HellaSwag validation 5294","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get your wedding party properly prepped to talk [title] Meet all party participants. [step] If you're the bride, groom, or maid of honor, provide some icebreakers, perhaps even a quick game to get everyone acquainted and more at ease. All the wedding events will be uncomfortable if you don't meet the other attendants.\",\"endings\":\"[\\\"Some photos may feel appropriate, but also you don't want to be out and about in the middle of a crowded city. [title] Get out your waterproof leather purses.\\\", 'Make sure that everyone you invite knows the etiquette of the occasion. Doing so should help and not affect the conversation.', 'If you aren\\\\'t formally introduced, make it a point to introduce yourself and explain how you know the bride or groom. Try, \\\" hi, i\\\\'m rick.', 'Still, getting to know the staff while jumping around can help out. [substeps] Serving a meal and getting to know various faces of the crowd can take a little time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}