{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0df4c45-4440-53ee-8a27-d7e0f3bde2b7","task_key":"default--validation--48050","task_revision_id":"1","upstream_id":"48050","short_description":"HellaSwag validation 48050","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to photograph a winter wedding [title] Pack your equipment the night before. [step] Charge all batteries, cameras, and laptops the night before the wedding. Also, bring backup equipment in case your main equipment breaks down while taking the photos.\",\"endings\":\"['Clothes, bring a blanket and vest as well as some light jackets as these can sometimes be bought at specialty stores. [substeps] For indoor weddings, keep your guests warm, but not too hot.', \\\"Equipment includes things like spare batteries, cameras, lenses and memory cards. Put everything in photo bags so it's ready to go in the morning.\\\", \\\"Then everything else goes into a different bag. You don't want to waste your trip on missing and misplaced gear.\\\", '[substeps] Remember that you can get help from everyone including the photographer if your budget is limited. Even if your budget is not dramatically reduced, you can still get extra headshots for your clients or for people at your wedding who are walking in to say goodbye.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}