{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e5ac431-cb20-5aeb-b799-66656227d6d9","task_key":"default--validation--44843","task_revision_id":"1","upstream_id":"44843","short_description":"HellaSwag validation 44843","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to socialize a dog with other dogs [title] Start with an interaction with a single dog. [step] A socially mature dog, which is a dog between one and three years, will not enjoy playing in a large group of dogs if it is already socialized. Something like a dog park would be overwhelming for your dog.\",\"endings\":\"['Trying to socialize a senior dog can be stressful. However, wait until your dog is fully grown or has one or two years of age before making your first introduction to your dog.', \\\"You should start with small, structured interactions with a single dog. Find a friend or family member with a dog that's appropriately socialized and take your dogs on a walk together.\\\", 'To overcome the immediate lack of socialization in a new dog, introduce it to others. [substeps] Say, \\\" hi fido, i\\\\'m human.', 'Showing your dog is also an introduction, rather than having it interact with another dog. [title] Choose a leash that works for you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}