{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19c70b4a-9416-548c-9c1c-39da0efec861","task_key":"default--validation--49514","task_revision_id":"1","upstream_id":"49514","short_description":"HellaSwag validation 49514","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to plan a vacation [title] Make a list of the top 5 places you want to travel. [step] If there are other people traveling with you, have them do the same thing. [title] Decide why you want to travel.\",\"endings\":\"[\\\"[step] In this position, you'll be less dependent on money. In fact, you won't much experience europe at all.\\\", \\\"[step] Because it's your only way to spend your vacation, there's no outside cause. Have them do it if they have young children or if they want to see their favorite museums or celebrities.\\\", \\\"[step] Is it for fun, fun, or to do something fun? Maybe just to save some money on food, or just to enjoy a hike? Decide what it is. The important thing about the trip is to be sure you really don't need another vacation ticket, unless this is going to be short-haul.\\\", '[step] If you know why you want to leave home, choosing a destination will be easier. Determining whether your goal is to relax and unwind, have new adventures, see famous or ancient sights, or provide your children with lifelong memories will determine what kind of destination you should choose.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}