{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b426e2b1-507f-57cf-9ed1-ba10d88f64ac","task_key":"default--validation--32857","task_revision_id":"1","upstream_id":"32857","short_description":"HellaSwag validation 32857","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make a \\\" draw my life \\\" video [title] Start a document either on physical pieces of paper or digitally on your computer to store all of your information. [step] This document will include any and all notable events in your life, so you will need plenty of paper if you are using physical paper. [title] Contact your family.\",\"endings\":\"['[step] Talk to them and ask for their support. Always try to have a \\\" family meeting.', \\\"[step] If you have any siblings who aren't related to any of your other relatives, call ahead and let them know you are creating the video. [substeps] Don't make it too obvious with the press as this can drive off the edge of your decision to create your video.\\\", '[step] Chances are, your family will have the most intimate knowledge of your life and can help describe who you are. Your parents, in particular, will remember what you were like as an infant and a young child much better than you can.', \\\"[step] Tell them your purpose for creating this video, and allow them to pitch in on what you are drawing. It's best to involve them beforehand or ask them to sing in and take the lead as they draw! [title] Make a silhouette of the subject.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}